diff --git a/CAPABILITIES.md b/CAPABILITIES.md index f66d6f0a44..c99b8c1be5 100644 --- a/CAPABILITIES.md +++ b/CAPABILITIES.md @@ -289,10 +289,11 @@ improvising or retrying when something is off. | See what your own agent gets wrong, with no risk: `openadapt shadow` is a loopback pass-through proxy beside the agent. It checks each write against the system of record (FHIR through `ensure_record`; SQL through the verifier's whole-database frame), checks each `CONFIRMED` again later, and writes `report.html` and `report.json`. It never blocks, changes or resends a write; the only delay is the read before a write (at most `hold_ms`) | [#1678](https://github.com/OpenAdaptAI/OpenAdapt/pull/1678) | Real app (Medplum 5.2.8, 37 cases through a fault proxy: scripted agent 18 of 18 harms caught, 0 correct writes refuted; Claude Sonnet 5.5 9 of 9 caught, 4 correct but corrected writes refuted; 0 writes by the shadow; 15 to 25 ms median added per write; setup 76.5 s machine time, 3 decisions); real app, SQL mode (Dolibarr 21 on MariaDB 11.4, 13 cases: 8 of 8 harms caught, 5 of 5 correct writes confirmed, 292 ms median added); synthetic (SQLite) | `openadapt shadow`, [SHADOW_MODE.md](packages/flow/docs/SHADOW_MODE.md), [SHADOW_RESULTS.md](packages/flow/benchmark/fhir_medplum/public/SHADOW_RESULTS.md) | | Skill cache for framework agents: an agent in any framework does a browser task through compile-agent's harness as plain tools (`BrowserTools`); when the system of record shows exactly the task's record and the operator confirms the checks, the run is canaried, admitted and emitted as an MCP server. A repeat of the same task (instruction, record, app, contract, environment) calls that tool instead of the agent: no model calls, the program's checks and this task's record check on every run, eviction after a failed run | On main | Synthetic (MockMed, LangGraph on `claude-sonnet-5-5`, 3 trials, repeats on the same record: run 1 6 model calls and $0.022; runs 2 to 10 27/27 VERIFIED, 0 model calls, 7.8 s median); Trac 1.6: run 1 verified, not admitted | `openadapt_flow.integrations.agents.skills`, [benchmark/skill_cache](packages/flow/benchmark/skill_cache/README.md) | | Verified write tools for agent frameworks: `verified(tool, contract=...)` wraps a Claude Agent SDK, OpenAI Agents SDK, LangGraph or CrewAI write tool in one line. The contract and any values the operator pins come from the operator, never the agent's arguments; the store is read before and after; the agent gets a receipt or a refusal, and a refused write is held instead of retried | On main | Synthetic (each framework's own agent loop with a scripted model, writing through the example app's order form over real HTTP: verified save, a save the app drops, a save it keeps twice); one real-model run per framework, opt-in | `openadapt_flow.integrations.agents`, extras `claude-agent-sdk`, `openai-agents`, `langgraph` (CrewAI: install `crewai` beside it); [AGENT_FRAMEWORKS.md](packages/flow/docs/AGENT_FRAMEWORKS.md) | -| Teach a browser write as a verified tool: with write contracts (`serve --teach-browser --contracts DIR`), the agent teaches a save once; run 1 is verified in the system of record before the `taught_` tool exists, and every call makes the write and returns a verified receipt or a refusal. Sign-in values come from the server's environment and never reach the agent or the program. Controls in named iframes are found through their frame | On main ([#1589](https://github.com/OpenAdaptAI/OpenAdapt/pull/1589)) | Synthetic (the example app's sign-in and order form in headless Chromium over MCP: verified save, a save the app drops, a save it keeps twice) | `openadapt-agent serve --teach-browser --contracts`, [VERIFY_WRITES.md](packages/flow/docs/VERIFY_WRITES.md#teach-a-write-as-a-tool) | +| Browser teach refuses taught writes: a write an agent teaches needs a person's signed teach authorization, and this lane can't bind one, so `teach_start` with a contract is refused and an existing write tool doesn't run. Writes are taught through `serve --teach --teach-authorization`. The read-only lane (downloads) is unchanged | On main ([#1589](https://github.com/OpenAdaptAI/OpenAdapt/pull/1589), [#1436](https://github.com/OpenAdaptAI/OpenAdapt/pull/1436)) | Unit test of the refusal | `openadapt-agent serve --teach-browser`, [TEACH_AUTHORIZATION.md](packages/flow/docs/TEACH_AUTHORIZATION.md) | | Guard an agent you already run: attach to its own Chromium over CDP (one added line), hold each save until the record on screen is the one the task names, block it and tell the agent why when it isn't, read the save back; a run that reads back correctly becomes a draft program with a record check from scaffold-verifier, which a person confirms and admits, then a `taught_*` tool with no model calls. `--observe-only` never blocks | On main ([#1357](https://github.com/OpenAdaptAI/OpenAdapt/pull/1357), stacked on [#1335](https://github.com/OpenAdaptAI/OpenAdapt/pull/1335)) | Synthetic (MockMed, scripted Playwright agent, `effect_e2e` faults) | `openadapt-flow guard`, [GUARD_AN_AGENT.md](packages/flow/docs/GUARD_AN_AGENT.md) | | Task ledger per job: rules locked when the job opens and enforced on every run, item status set only from OpenAdapt's own results, and yes-or-no questions a person answers on the phone or in Desktop, including when a run stops | On main ([#1332](https://github.com/OpenAdaptAI/OpenAdapt/pull/1332)) | Synthetic (30-item MockMed worklist, scripted agent, no model calls) | MCP tools `open_job`, `get_job`, `add_items`, `record_evidence`, `ask_user`, `get_answer`, `change_constraint`, `close_job`; [Track a long job](packages/agent/README.md#track-a-long-job) | | Limit a teaching agent to the declared task on every teach path (`compile_agent`, `compile-agent`, teach over MCP, the airbag guard): typed text must be a declared value, links and browser requests to other origins are refused, writes are checked before they are sent, and the record is read back before a draft exists. Injected instructions in record notes, banners, tooltips, alt text and server responses are blocked or caught on MockMed through each path (scripted agent; a live-model mode is off by default). Self-repair refuses a control that leaves the app | On main ([#1359](https://github.com/OpenAdaptAI/OpenAdapt/pull/1359), [#1386](https://github.com/OpenAdaptAI/OpenAdapt/pull/1386)) | Synthetic | `openadapt_flow.teach_scope`, [THREAT_MODEL.md](packages/flow/docs/THREAT_MODEL.md) | +| Teach authorization: a person signs what a teaching agent may do (task, origins, inputs, the one write, forbidden actions and controls) with their approver key, and every teach path is bound to it or refused. Verified with the approver roster's own verifier, so an unknown, removed or wrong key, an unpinned root, a changed statement or another task refuses. Admission of every JIT draft needs one (`teach_auth_not_recorded` otherwise), as do an agent-demonstrated fix's approval and self-repair of an agent-taught program; the airbag handoff's drafts and browser-teach writes are refused | On main ([#1436](https://github.com/OpenAdaptAI/OpenAdapt/pull/1436)) | Synthetic (real approver keys, real roster, real admission) | `openadapt-flow teach-authorization`, [TEACH_AUTHORIZATION.md](packages/flow/docs/TEACH_AUTHORIZATION.md) | | Compile an agent's first verified run from a prompt (`compile-agent`), and a code-as-action authoring agent | On main ([#1277](https://github.com/OpenAdaptAI/OpenAdapt/pull/1277), [#1305](https://github.com/OpenAdaptAI/OpenAdapt/pull/1305)); the agent harness behind it runs on every backend, on main ([#1283](https://github.com/OpenAdaptAI/OpenAdapt/pull/1283)) | Synthetic | `compile-agent` | | Score harness optimizers (Harness², Meta-Harness) on GUI tasks with the record read-back as the reward, with full trace export | On main ([#1299](https://github.com/OpenAdaptAI/OpenAdapt/pull/1299), [#1300](https://github.com/OpenAdaptAI/OpenAdapt/pull/1300), [#1301](https://github.com/OpenAdaptAI/OpenAdapt/pull/1301)) | Synthetic (scripted proposer) | `integrations/harness2/` | | One scoreboard comparing the agent alone with OpenAdapt across tasks (correctness by read-back, cost, time, items per hour) | On main ([#1303](https://github.com/OpenAdaptAI/OpenAdapt/pull/1303)) | Synthetic and local OpenEMR and openIMIS test copies | `benchmark/scoreboard/` | @@ -363,6 +364,7 @@ A test fails when a command has no file or this list is out of date. - `propose-qualification` - `qualify` - `scaffold-verifier` +- `teach-authorization` **Run and answer:** diff --git a/llms-full.txt b/llms-full.txt index 803971e8f2..99efafab06 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -692,10 +692,11 @@ improvising or retrying when something is off. | See what your own agent gets wrong, with no risk: `openadapt shadow` is a loopback pass-through proxy beside the agent. It checks each write against the system of record (FHIR through `ensure_record`; SQL through the verifier's whole-database frame), checks each `CONFIRMED` again later, and writes `report.html` and `report.json`. It never blocks, changes or resends a write; the only delay is the read before a write (at most `hold_ms`) | [#1678](https://github.com/OpenAdaptAI/OpenAdapt/pull/1678) | Real app (Medplum 5.2.8, 37 cases through a fault proxy: scripted agent 18 of 18 harms caught, 0 correct writes refuted; Claude Sonnet 5.5 9 of 9 caught, 4 correct but corrected writes refuted; 0 writes by the shadow; 15 to 25 ms median added per write; setup 76.5 s machine time, 3 decisions); real app, SQL mode (Dolibarr 21 on MariaDB 11.4, 13 cases: 8 of 8 harms caught, 5 of 5 correct writes confirmed, 292 ms median added); synthetic (SQLite) | `openadapt shadow`, [SHADOW_MODE.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/SHADOW_MODE.md), [SHADOW_RESULTS.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/benchmark/fhir_medplum/public/SHADOW_RESULTS.md) | | Skill cache for framework agents: an agent in any framework does a browser task through compile-agent's harness as plain tools (`BrowserTools`); when the system of record shows exactly the task's record and the operator confirms the checks, the run is canaried, admitted and emitted as an MCP server. A repeat of the same task (instruction, record, app, contract, environment) calls that tool instead of the agent: no model calls, the program's checks and this task's record check on every run, eviction after a failed run | On main | Synthetic (MockMed, LangGraph on `claude-sonnet-5-5`, 3 trials, repeats on the same record: run 1 6 model calls and $0.022; runs 2 to 10 27/27 VERIFIED, 0 model calls, 7.8 s median); Trac 1.6: run 1 verified, not admitted | `openadapt_flow.integrations.agents.skills`, [benchmark/skill_cache](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/benchmark/skill_cache/README.md) | | Verified write tools for agent frameworks: `verified(tool, contract=...)` wraps a Claude Agent SDK, OpenAI Agents SDK, LangGraph or CrewAI write tool in one line. The contract and any values the operator pins come from the operator, never the agent's arguments; the store is read before and after; the agent gets a receipt or a refusal, and a refused write is held instead of retried | On main | Synthetic (each framework's own agent loop with a scripted model, writing through the example app's order form over real HTTP: verified save, a save the app drops, a save it keeps twice); one real-model run per framework, opt-in | `openadapt_flow.integrations.agents`, extras `claude-agent-sdk`, `openai-agents`, `langgraph` (CrewAI: install `crewai` beside it); [AGENT_FRAMEWORKS.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/AGENT_FRAMEWORKS.md) | -| Teach a browser write as a verified tool: with write contracts (`serve --teach-browser --contracts DIR`), the agent teaches a save once; run 1 is verified in the system of record before the `taught_` tool exists, and every call makes the write and returns a verified receipt or a refusal. Sign-in values come from the server's environment and never reach the agent or the program. Controls in named iframes are found through their frame | On main ([#1589](https://github.com/OpenAdaptAI/OpenAdapt/pull/1589)) | Synthetic (the example app's sign-in and order form in headless Chromium over MCP: verified save, a save the app drops, a save it keeps twice) | `openadapt-agent serve --teach-browser --contracts`, [VERIFY_WRITES.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/VERIFY_WRITES.md#teach-a-write-as-a-tool) | +| Browser teach refuses taught writes: a write an agent teaches needs a person's signed teach authorization, and this lane can't bind one, so `teach_start` with a contract is refused and an existing write tool doesn't run. Writes are taught through `serve --teach --teach-authorization`. The read-only lane (downloads) is unchanged | On main ([#1589](https://github.com/OpenAdaptAI/OpenAdapt/pull/1589), [#1436](https://github.com/OpenAdaptAI/OpenAdapt/pull/1436)) | Unit test of the refusal | `openadapt-agent serve --teach-browser`, [TEACH_AUTHORIZATION.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/TEACH_AUTHORIZATION.md) | | Guard an agent you already run: attach to its own Chromium over CDP (one added line), hold each save until the record on screen is the one the task names, block it and tell the agent why when it isn't, read the save back; a run that reads back correctly becomes a draft program with a record check from scaffold-verifier, which a person confirms and admits, then a `taught_*` tool with no model calls. `--observe-only` never blocks | On main ([#1357](https://github.com/OpenAdaptAI/OpenAdapt/pull/1357), stacked on [#1335](https://github.com/OpenAdaptAI/OpenAdapt/pull/1335)) | Synthetic (MockMed, scripted Playwright agent, `effect_e2e` faults) | `openadapt-flow guard`, [GUARD_AN_AGENT.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/GUARD_AN_AGENT.md) | | Task ledger per job: rules locked when the job opens and enforced on every run, item status set only from OpenAdapt's own results, and yes-or-no questions a person answers on the phone or in Desktop, including when a run stops | On main ([#1332](https://github.com/OpenAdaptAI/OpenAdapt/pull/1332)) | Synthetic (30-item MockMed worklist, scripted agent, no model calls) | MCP tools `open_job`, `get_job`, `add_items`, `record_evidence`, `ask_user`, `get_answer`, `change_constraint`, `close_job`; [Track a long job](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/agent/README.md#track-a-long-job) | | Limit a teaching agent to the declared task on every teach path (`compile_agent`, `compile-agent`, teach over MCP, the airbag guard): typed text must be a declared value, links and browser requests to other origins are refused, writes are checked before they are sent, and the record is read back before a draft exists. Injected instructions in record notes, banners, tooltips, alt text and server responses are blocked or caught on MockMed through each path (scripted agent; a live-model mode is off by default). Self-repair refuses a control that leaves the app | On main ([#1359](https://github.com/OpenAdaptAI/OpenAdapt/pull/1359), [#1386](https://github.com/OpenAdaptAI/OpenAdapt/pull/1386)) | Synthetic | `openadapt_flow.teach_scope`, [THREAT_MODEL.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/THREAT_MODEL.md) | +| Teach authorization: a person signs what a teaching agent may do (task, origins, inputs, the one write, forbidden actions and controls) with their approver key, and every teach path is bound to it or refused. Verified with the approver roster's own verifier, so an unknown, removed or wrong key, an unpinned root, a changed statement or another task refuses. Admission of every JIT draft needs one (`teach_auth_not_recorded` otherwise), as do an agent-demonstrated fix's approval and self-repair of an agent-taught program; the airbag handoff's drafts and browser-teach writes are refused | On main ([#1436](https://github.com/OpenAdaptAI/OpenAdapt/pull/1436)) | Synthetic (real approver keys, real roster, real admission) | `openadapt-flow teach-authorization`, [TEACH_AUTHORIZATION.md](https://github.com/OpenAdaptAI/OpenAdapt/blob/main/packages/flow/docs/TEACH_AUTHORIZATION.md) | | Compile an agent's first verified run from a prompt (`compile-agent`), and a code-as-action authoring agent | On main ([#1277](https://github.com/OpenAdaptAI/OpenAdapt/pull/1277), [#1305](https://github.com/OpenAdaptAI/OpenAdapt/pull/1305)); the agent harness behind it runs on every backend, on main ([#1283](https://github.com/OpenAdaptAI/OpenAdapt/pull/1283)) | Synthetic | `compile-agent` | | Score harness optimizers (Harness², Meta-Harness) on GUI tasks with the record read-back as the reward, with full trace export | On main ([#1299](https://github.com/OpenAdaptAI/OpenAdapt/pull/1299), [#1300](https://github.com/OpenAdaptAI/OpenAdapt/pull/1300), [#1301](https://github.com/OpenAdaptAI/OpenAdapt/pull/1301)) | Synthetic (scripted proposer) | `integrations/harness2/` | | One scoreboard comparing the agent alone with OpenAdapt across tasks (correctness by read-back, cost, time, items per hour) | On main ([#1303](https://github.com/OpenAdaptAI/OpenAdapt/pull/1303)) | Synthetic and local OpenEMR and openIMIS test copies | `benchmark/scoreboard/` | @@ -766,6 +767,7 @@ A test fails when a command has no file or this list is out of date. - `propose-qualification` - `qualify` - `scaffold-verifier` +- `teach-authorization` **Run and answer:** diff --git a/packages/agent/tests/test_repair_e2e.py b/packages/agent/tests/test_repair_e2e.py index bac4952395..717dc64515 100644 --- a/packages/agent/tests/test_repair_e2e.py +++ b/packages/agent/tests/test_repair_e2e.py @@ -95,7 +95,33 @@ def refresh(self, entry, run_id): return None -def test_halt_teach_approve_replay(tmp_path: Path) -> None: +def _teach_authority(tmp_path: Path, monkeypatch: pytest.MonkeyPatch): + """A person on a real approver roster signs that an agent may repair it.""" + + from openadapt_flow.teach_authorization_fixture import LocalAuthorizer + + keys = tmp_path / "keys" + person = LocalAuthorizer(keys, "clinic-lead") + statement = person.issue( + tmp_path / "teach-authorization", + { + # The program's own workflow name, which approval checks. + "task": {"task_id": fx.WORKFLOW_NAME, "summary": "Repair the save step"}, + "purposes": ["repair"], + "origins": ["http://127.0.0.1:9"], + "writes": { + "record_kind": "encounter", + "identity": {"type": {"value": "encounter"}}, + }, + }, + ) + # Approval checks it again against the roster the operator pins. + monkeypatch.setenv("OPENADAPT_JIT_KEY_DIR", str(keys)) + monkeypatch.setenv("OPENADAPT_APPROVER_ROOT", person.root) + return person.authority(statement) + + +def test_halt_teach_approve_replay(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: bundle = fx.base_bundle(tmp_path / "bundle") access = tmp_path / "access.json" set_access(access, workflow=WORKFLOW, mode="scrubbed_only", enabled_by="admin") @@ -106,6 +132,7 @@ def test_halt_teach_approve_replay(tmp_path: Path) -> None: demonstrator=screen, scrubber_factory=Scrubber, agent_id=AGENT, + teach_authorization=_teach_authority(tmp_path, monkeypatch), ) bridge = AgentBridge( None, RunnerConfig(runs_dir=tmp_path / "runs"), allow_run=True, repair_desk=desk @@ -173,6 +200,7 @@ def run(request_id: str) -> dict: assert provenance.screens_shared == 1 assert provenance.screens_scrubbed is True assert provenance.scrub_evidence.provider == "test" + assert provenance.teach_authorization["authorizer"] == "clinic-lead" # 4. The agent can't review or approve its own repair, even as a "human". with pytest.raises(RepairLifecycleError): diff --git a/packages/flow/CHANGELOG.md b/packages/flow/CHANGELOG.md index ec21cf1c0c..024c848a21 100644 --- a/packages/flow/CHANGELOG.md +++ b/packages/flow/CHANGELOG.md @@ -19,6 +19,10 @@ ### Features +- Teach authorizations (`openadapt.teach-authorization/v1`). A person signs, with an approver key on the signed approver roster, what an agent may do when it teaches or repairs a program: the task, origins, inputs, the one write and forbidden actions and controls. `openadapt-flow teach-authorization create|sign|verify`; `--teach-authorization` on `compile-agent` and `openadapt-agent serve --teach`. Design and the upgrade notes: `docs/TEACH_AUTHORIZATION.md`. + + **Breaking:** admission of every JIT draft now needs one (`teach_auth_not_recorded` otherwise), in every admission mode; so does a self-repair of an agent-taught program and the approval of a calling agent's fix. Drafts from the airbag handoff are refused, and browser teach (`serve --teach-browser`) no longer teaches or runs write tools. + - `run --self-repair` (and repair by default) now works outside a browser. `repair.session` in `deploy.yaml` says how to start the app for each probe, fault probe and resumed run (an argv, no shell, kept running while the app is up; `fault_commands`, `close_command`, `ready_s`). The run's configured backend must bind to a window owned by that command's process group, so a session never binds to the halted session or a copy already open; Flow ends the group afterwards, and closes the halted session before each probe. This works where the bound window is on Flow's machine (macOS, and the local RDP or Citrix client window); Windows over its agent and FreeRDP are refused for now. Without `repair.session`, a desktop run refuses repair with that reason, as before. - Signed write receipts. A verifier verdict, with its scope (stores read, time window, strength) and a plain list of what it did not check, is signed as a COSE_Sign1 in the IETF SCITT shape and appended to a local transparency log. `openadapt-flow receipt verify FILE --issuer-key --log-key` checks one offline and prints the not-checked list every time; a receipt that claims more than its scope is neither issued nor accepted. `verify_write(..., write_receipt=True)`, `issue_write_receipt(result)` and `openadapt-flow shadow --receipts` issue them; `receipt anchor` puts the digest in Rekor. Extra: `openadapt-flow[receipts]`. Design: `docs/WRITE_RECEIPTS.md`. diff --git a/packages/flow/benchmark/code_as_action/measure_mockmed.py b/packages/flow/benchmark/code_as_action/measure_mockmed.py index 09676ae114..60b3bafcb3 100644 --- a/packages/flow/benchmark/code_as_action/measure_mockmed.py +++ b/packages/flow/benchmark/code_as_action/measure_mockmed.py @@ -37,6 +37,8 @@ import common # noqa: E402 +from openadapt_flow.teach_authorization_fixture import demo_authority + PROMPT = ( "Open the referral for Jane Sample, start a Triage encounter, and save " "this note: {note}" @@ -149,6 +151,17 @@ def main() -> int: load_before = common.load_1m() started = time.time() try: + # A throwaway approver signs what run 1 may do. + compiler.teach_authorization = demo_authority( + work, + task_id=compiler.task.task_id, + url=base, + record={"patient_id": "p1", "type": "Triage"}, + inputs={"note": note}, + fields={"note": note}, + constants=["Jane Sample", "MRN P1"], + record_kind="encounter", + ) result = compiler.record(agent, {"note": note}) status, detail, run1 = result.status, result.detail, result.run1 except Exception as exc: # noqa: BLE001 - a failed run is a row diff --git a/packages/flow/benchmark/compile_agent/measure.py b/packages/flow/benchmark/compile_agent/measure.py index 1e2035cc22..3dfd8dbb26 100644 --- a/packages/flow/benchmark/compile_agent/measure.py +++ b/packages/flow/benchmark/compile_agent/measure.py @@ -36,6 +36,8 @@ from pathlib import Path from typing import Any +from openadapt_flow.teach_authorization_fixture import demo_authority + HERE = Path(__file__).resolve().parent RESULTS = HERE / "results" PROMPT = ( @@ -262,6 +264,17 @@ def main() -> int: # with one note for every trial, later runs read earlier runs' # screenshots from the prompt cache and cost less than a real run. note = f"{NOTE}, trial {trial}" + # A throwaway approver signs what run 1 may do. + compiler.teach_authorization = demo_authority( + work, + task_id=compiler.task.task_id, + url=base, + record={"patient_id": "p1", "type": "Triage"}, + inputs={"note": note}, + fields={"note": note}, + constants=["Jane Sample", "MRN P1"], + record_kind="encounter", + ) result = compiler.record(agent, {"note": note}) run1 = _clean_run1(result.run1) cost = float(((run1.get("agent") or {}).get("cost_usd")) or 0.0) diff --git a/packages/flow/benchmark/scoreboard/run.py b/packages/flow/benchmark/scoreboard/run.py index 015e91d8f0..6e580d036a 100644 --- a/packages/flow/benchmark/scoreboard/run.py +++ b/packages/flow/benchmark/scoreboard/run.py @@ -38,6 +38,7 @@ HERE = Path(__file__).resolve().parent sys.path.insert(0, str(HERE.parent)) +from openadapt_flow.teach_authorization_fixture import demo_authority # noqa: E402 from scoreboard import schema # noqa: E402 from scoreboard.agent import ( # noqa: E402 SYSTEM_TEMPLATE, @@ -405,6 +406,16 @@ def _mockmed_jit( key_dir=cache / "admission-keys", model_key=model_key() if agent is not None else None, dialect=dialect, + # A throwaway approver stands in for the person who signs what run 1 + # may do; admission needs it. + teach_authorization=demo_authority( + cache, + task_id=mockmed.TASK_ID, + url=server.url(it), + record={"note": it.values["note"]}, + inputs=it.values, + record_kind="encounter", + ), ) @@ -446,6 +457,14 @@ def _openemr_jit( key_dir=cache / "admission-keys", model_key=model_key() if agent is not None else None, dialect=dialect, + teach_authorization=demo_authority( + cache, + task_id=T.TASK_ID, + url=T.entry_url(), + record={"fname": values["fname"], "lname": values["lname"]}, + inputs=values, + record_kind="patient", + ), ) diff --git a/packages/flow/benchmark/silent_saves/compile_arm.py b/packages/flow/benchmark/silent_saves/compile_arm.py index 79026d4ec3..57d00d99a8 100644 --- a/packages/flow/benchmark/silent_saves/compile_arm.py +++ b/packages/flow/benchmark/silent_saves/compile_arm.py @@ -25,6 +25,8 @@ from pathlib import Path from typing import Any, Optional +from openadapt_flow.teach_authorization_fixture import demo_authority + from . import spend from .executors import BENCHMARK_DIR, Report, Task @@ -99,6 +101,16 @@ def _jit(self, task: Task, agent: Any = None) -> Any: key_dir=self.cache / "admission-keys", model_key=self._sb.model_key() if agent is not None else None, dialect="anthropic", + # A throwaway approver stands in for the person who signs what + # run 1 may do; admission needs it. + teach_authorization=demo_authority( + self.cache, + task_id=TASK_ID, + url=task.url, + record={"note": task.params["note"]}, + inputs=task.params, + record_kind="encounter", + ), ) # -- admission, scripted -------------------------------------------------- diff --git a/packages/flow/benchmark/silent_saves/executors.py b/packages/flow/benchmark/silent_saves/executors.py index aa36517971..5658758e8d 100644 --- a/packages/flow/benchmark/silent_saves/executors.py +++ b/packages/flow/benchmark/silent_saves/executors.py @@ -303,6 +303,7 @@ def run(self, task: Task) -> Report: cache_dir=self.cache, key_dir=self.cache / "admission-keys", dialect="anthropic", + teach_authorization=_demo_authority(self.cache, task), ) try: result = jit.run(**task.params) @@ -376,3 +377,22 @@ def _python(target: str) -> Any: if not isinstance(obj, Executor): raise SystemExit(f"{target} has no name and run(task) -> Report") return obj + + +def _demo_authority(cache: Path, task: Any) -> Any: + """A throwaway approver signs what run 1 of this task may do. + + Admission needs a person's teach authorization; a benchmark stands one + in (openadapt_flow.teach_authorization_fixture). The note is the record. + """ + + from openadapt_flow.teach_authorization_fixture import demo_authority + + return demo_authority( + cache, + task_id="mockmed.add_note", + url=task.url, + record={"note": task.params["note"]}, + inputs=task.params, + record_kind="encounter", + ) diff --git a/packages/flow/benchmark/skill_cache/measure.py b/packages/flow/benchmark/skill_cache/measure.py index 71731e2dab..517c228ed0 100644 --- a/packages/flow/benchmark/skill_cache/measure.py +++ b/packages/flow/benchmark/skill_cache/measure.py @@ -33,6 +33,8 @@ from pathlib import Path from typing import Any +from openadapt_flow.teach_authorization_fixture import demo_authority + HERE = Path(__file__).resolve().parent MODEL = "claude-sonnet-5-5" TRACE: Path | None = None @@ -259,13 +261,13 @@ def main() -> int: args.work.mkdir(parents=True, exist_ok=True) TRACE = args.work / "agent-tool-calls.jsonl" + import subprocess + from langchain_anthropic import ChatAnthropic from openadapt_flow.integrations.agents.skills import SkillCache from openadapt_flow.jit.keys import load_customer_model_key - import subprocess - commit = subprocess.run( ["git", "rev-parse", "--short=10", "HEAD"], capture_output=True, @@ -313,6 +315,15 @@ def offer(proposal: dict) -> str: # on p1 a duplicate, so it is reset before every task. reset() before = load_1m() + # A throwaway approver signs what run 1 of this task may do. + cache.common["teach_authorization"] = demo_authority( + args.work, + task_id=cache.task_id(app["prompt"], values), + url=app["url"], + record=app["expect"], + inputs=values, + constants=app["on_screen"], + ) result = cache.run(app["prompt"], values, agent) row = { "app": args.app, diff --git a/packages/flow/docs/TEACH_AUTHORIZATION.md b/packages/flow/docs/TEACH_AUTHORIZATION.md new file mode 100644 index 0000000000..f0a85ed057 --- /dev/null +++ b/packages/flow/docs/TEACH_AUTHORIZATION.md @@ -0,0 +1,141 @@ +# Teach authorization + +A teach authorization (`openadapt.teach-authorization/v1`) is a short +statement a person signs before an agent teaches or repairs a program. It +names what the agent may do: + +- the task (`task_id` and a one-line summary); +- the app origins the browser may reach; +- the inputs the agent may type, each spelled out or given as a SHA-256 the + operator's value must match; +- the one permitted write: the record kind, the identity of the intended + record, the fields and values the save may write, and how many rows; +- forbidden actions (drag, right click, a `DELETE` request, a new window, a + download, a file chooser) and forbidden controls by label; +- whether it covers `teach`, `repair` or both, and when it expires. + +The teach scope (`openadapt_flow.teach_scope`, see +[THREAT_MODEL.md](THREAT_MODEL.md)) is then bound from the signed statement, +not from the agent's own description of the task. Admission checks the +signature again. The module is `openadapt_flow.teach_authorization`. + +## Who signs, and how it is checked + +The signer is a person on the signed approver roster, with the same approver +key that answers saves ([APPROVAL_MODES.md](APPROVAL_MODES.md#the-approver-key)). +The signature is an approver attestation over the canonical statement, kept +beside it as `teach-authorization.json.sig`. Verification uses the approver +verifier (`approval_mode.human_attestation.refusal`): + +- The caller pins the approver root and the key directory that holds the + roster. Commands a person runs (`jit admit`, `compile-agent`, + `teach-authorization`) take `--approver-root` or `OPENADAPT_APPROVER_ROOT`, + and `--key-dir` or `OPENADAPT_JIT_KEY_DIR`; repair approval reads the two + environment variables. A + server the agent runs (`openadapt-agent serve --teach`, its repair desk) + takes them only from the deployment config's `approvers` section + (`--config`): its root, key directory and recorded head. Without a pin, + nothing verifies. +- The signer must be on that roster when the check runs, under the + statement's `authorizer` name, with the statement's `authorizer_public_key`. + A key that was never added refuses (`teach_auth_unknown_signer`), and so + does one an approver removed since (`approver remove`). The roster must + extend every head recorded earlier, so an older roster put back refuses. +- A changed statement refuses (`teach_auth_bad_signature`). The signature + is over its own domain (`openadapt.teach-authorization/v1`), so a signed + save approval or roster change never counts as one. +- The roster never holds an agent: a principal that names an agent or model, + or a key that holds another role, is not an approver. Admission also + refuses a statement signed by the agent that recorded the draft. + +## Register a key and sign + +The signer uses an encrypted approver key file, on any operating system; +no Secure Enclave is needed: + +```bash +export OPENADAPT_JIT_KEY_PASSPHRASE=... # never on the command line +openadapt-flow jit keygen --out clinic-lead.pem +# An approver already on the roster adds it (or start a roster with it: +# openadapt-flow jit approver init --principal clinic-lead --key clinic-lead.pem) +openadapt-flow jit approver add --principal clinic-lead \ + --public-key PUBLIC_KEY_HEX --signer-key ADMIN.pem --approver-root ROOT_HEX +``` + +On Windows and Linux, `openadapt-desktop approver-key setup` also creates +such a key file (its passphrase in the OS keychain) and prints the same +`jit approver` commands. Desktop's Touch ID key on a Mac answers item +approvals only; it doesn't sign teach authorizations. + +Then write and sign the statement: + +```bash +openadapt-flow teach-authorization create spec.json --out auth/ \ + --authorizer clinic-lead --authorizer-public-key PUBLIC_KEY_HEX +openadapt-flow teach-authorization sign auth/ --approver-key clinic-lead.pem \ + --approver-root ROOT_HEX +openadapt-flow teach-authorization verify auth/ --approver-root ROOT_HEX +``` + +A statement expires 24 hours after `create` unless `--expires-in-hours` +(or `expires_at` in the Python API) says otherwise, at most 90 days on. `sign` shows what the statement allows and +needs a person at an interactive terminal who types `yes`. + +## Where it is required + +| Entry point | How it is bound | Without one | +| --- | --- | --- | +| JIT capture (`compile_agent(..., teach_authorization=TeachAuthority(...))`) | The capture's scope is bound from the statement; a `teach_scope` may only narrow it. The statement is kept beside the draft, and its digest is in the signed recorder block | Admission refuses (`teach_auth_not_recorded`) | +| `compile-agent` (`--teach-authorization`, `FirstRunCompiler`, `compile_first_run`, `SkillCache`) | Run 1's scope is the statement's alone; the command's `--expect`, `--on-screen` and `--allow-text` don't widen it. For `SkillCache`, the statement names `SkillCache.task_id(prompt, params)` | Admission refuses | +| Teach over MCP (`serve --teach --teach-authorization`; `teach_act` and `teach_run`) | Run 1 runs under the statement's scope, under its task id; the agent's declared inputs must be the statement's. `teach_run`'s scripts act through the same run 1, so every action and the save are checked against the same scope | Admission (`teach_admit`, `compile-agent admit`) refuses | +| Browser teach writes (`serve --teach-browser`, `teach_start` with a contract) | Not bound: this lane has no teach scope guard to bind the statement to | Refused: no write tool is taught, and a write tool taught before doesn't run. The read-only lane sends no write and is unchanged | +| Agent-demonstrated fix (`teach_repair`, `serve --teach-authorization`) | Before the click, the desk verifies a statement that covers `repair` and names the program's workflow name as its task; the repair store keeps it. Approval checks it again, for that workflow | The fix is proposed, and `RepairStore.approve` refuses it | +| Self-repair of an agent-taught program (`admit_repair`) | The prior admission's authorization must cover `repair` and verify now; its statement is copied into the repaired bundle | The repair's admission (probe or full) refuses | +| Airbag handoff (`agent_guard.handoff`) | Not bound: the guarded run 1 runs under the guard's task file, and the handoff can't attach a person's signature to it yet | Admission refuses its draft (`teach_auth_not_recorded`). Teach the task through `compile-agent` or `serve --teach` with an authorization instead | + +Every JIT draft is compiled from an agent's run, so admission requires the +authorization for every JIT draft, in each admission mode (a person, an +admitter key, an agent under a signed policy). Admission reads the signed +recorder block, not the editable candidate fields, and checks: the kept +statement's digest equals the recorded one; the signature verifies against +the roster now; the task is the draft's; the capture ran under the scope the +statement binds, and its read-back verified. + +## Upgrading + +Programs and tools made before teach authorizations carry none, so: + +- a JIT draft waiting for admission is refused. Teach it again with an + authorization. +- an admitted JIT program still runs, but a self-repair of it is refused + until a person teaches it again with an authorization that covers + `repair` and admits it. +- a write tool taught in browser teach doesn't run. +- the benchmarks that compile an agent's run and `examples/jit/run_mockmed.py` + now sign a throwaway authorization for each run + (`teach_authorization_fixture.demo_authority`), standing in for the person. + +## Limits + +- The approver root comes from the caller: a flag, the environment or a + deployment config. A process that controls both the environment and the + key directory can pin its own roster. Keep the key directory and the + approver keys away from the agents they govern, as for approvals. +- Self-repair's carried admission (`run --self-repair`) and repair approval + take the root from `OPENADAPT_APPROVER_ROOT` and the roster from + `OPENADAPT_JIT_KEY_DIR`, not from a governed run's deployment config. + Without them they refuse. +- `serve --teach` reads the roster from the config's `approvers.key_dir` + when it names one, else from `OPENADAPT_JIT_KEY_DIR`. Pin `approvers.head` + in the config too, or a copied older roster in that folder isn't caught. +- Self-repair of a program a person demonstrated (not a JIT draft) is a + repair a person reviews; it carries no teach authorization. +- A statement may be used for any number of teach runs of its task until it + expires; its nonce is not spent. Revoking the signer stops new admissions + and repairs, not programs already admitted. +- Forbidden controls are matched by their accessible label, on web pages + only; a click whose label can't be read is refused. Keyboard activation of + a focused control is not checked against the list. +- An agent-demonstrated fix is recognized by its provenance, a JSON file in + the repair store that the store doesn't sign. A process that can write the + store can strip it, and the approval then asks for no authorization. diff --git a/packages/flow/docs/THREAT_MODEL.md b/packages/flow/docs/THREAT_MODEL.md index 6d734cb9cf..9a8cad59a1 100644 --- a/packages/flow/docs/THREAT_MODEL.md +++ b/packages/flow/docs/THREAT_MODEL.md @@ -91,6 +91,12 @@ query strings). A clean scoped capture records the same report under `teach_scope` in the draft's candidate, so the reviewer sees that nothing was blocked and the read-back verified. +A scope written by the agent's harness is only as trustworthy as the +harness. A teach authorization ([TEACH_AUTHORIZATION.md](TEACH_AUTHORIZATION.md)) +moves the declaration to a person: the scope is bound from a statement they +sign with their approver key, and admission refuses an agent-taught draft +without one. + Every teach path passes a scope: | Path | Where the declaration comes from | What is enforced | diff --git a/packages/flow/docs/VERIFY_WRITES.md b/packages/flow/docs/VERIFY_WRITES.md index f71443376a..2888b1e5f7 100644 --- a/packages/flow/docs/VERIFY_WRITES.md +++ b/packages/flow/docs/VERIFY_WRITES.md @@ -98,21 +98,10 @@ A verified result proves the system of record holds what was declared. It doesn' ## Teach a write as a tool -`openadapt-agent serve --teach-browser --contracts ./contracts` lets your agent teach a browser write once, against a contract, and call it again as a `taught_` tool. Each call makes the write and returns the same verified receipt or refusal. - -For a taught write, the contract also lists the requests the write may send: - -```yaml -writes: ["POST /login", "POST /orders"] # the sign-in and the save -``` - -The browser stops any other write before it is sent, while teaching and on every call. The tool sends exactly the writes it was taught, in order and with the same fields, and a call where the page tries anything else ends in `needs_review`, never `done`. - -Sign-in values stay with the server. They go only into a password field or another field of a sign-in form, never into an address, and no form of them (encoded or not) reaches the agent: - -```bash -openadapt-agent serve --teach-browser --contracts ./contracts \ - --sign-in username=PORTAL_USER --sign-in password=PORTAL_PASSWORD -``` - -The example app has a sign-in and a New Order form to try it on: `openadapt agent example-app`, with user `clerk` and password `portal-demo`. +A write an agent teaches needs a person's signed teach authorization +([TEACH_AUTHORIZATION.md](TEACH_AUTHORIZATION.md)). Browser teach +(`openadapt-agent serve --teach-browser`) can't bind one, so it refuses a +`teach_start` that names a write contract, and a write tool taught there +earlier doesn't run. Teach the write with `openadapt-agent serve --teach +--teach-authorization STATEMENT`: run 1 runs under the signed scope, and the +program is admitted only with the signature. The read-only lane is unchanged. diff --git a/packages/flow/docs/commands/02-check-and-admit/teach-authorization.md b/packages/flow/docs/commands/02-check-and-admit/teach-authorization.md new file mode 100644 index 0000000000..e69de29bb2 diff --git a/packages/flow/docs/phi_at_rest.md b/packages/flow/docs/phi_at_rest.md index aa6dd7d295..c548354ec1 100644 --- a/packages/flow/docs/phi_at_rest.md +++ b/packages/flow/docs/phi_at_rest.md @@ -98,6 +98,7 @@ of truth; a test keeps this table in step with them. | `undo-lineage.json`, `record-check-binding.json` | Prior digests; record-check coverage | Encrypted | | `qualification-proposal.json`, `qualification-proposal.draft.json`, `qualification-admission.local-dev.json`, `qualification/*` | Qualification pins, cases and reports | Encrypted | | `admission-decision.json`, `admission.json`, `admit.json`, `jit_draft.json` | Admission records; some carry read-back values or a login name | Encrypted | +| `teach-authorization/*` | The person-signed teach authorization the program was taught under; a spelled-out input or record value is in it | Encrypted | | `openadapt-card.json`, `repair/*`, `evidence/*` | Operator card, repair candidates with anchor text, copied evidence | Encrypted | | Anything else | Unknown | `seal` refuses | diff --git a/packages/flow/examples/jit/run_mockmed.py b/packages/flow/examples/jit/run_mockmed.py index 4257c91eb8..979996f5ff 100644 --- a/packages/flow/examples/jit/run_mockmed.py +++ b/packages/flow/examples/jit/run_mockmed.py @@ -97,8 +97,22 @@ def wrapped(fault: str = "ok"): policy="clinical-write", cache_dir=cache, key_dir=cache / "admission-keys", + # A throwaway approver stands in for the person who signs + # what the agent may do (docs/TEACH_AUTHORIZATION.md). + teach_authorization=authority, ) + from openadapt_flow.teach_authorization_fixture import demo_authority + + authority = demo_authority( + cache, + task_id=TASK.task_id, + url=base, + record={"note": agent.inputs["note"]}, + inputs={"note": agent.inputs["note"]}, + record_kind="encounter", + ) + jit = wrapped() print("Run 1: the agent does the task while Flow records it") first = jit.run(note=agent.inputs["note"]) diff --git a/packages/flow/openadapt_flow/__main__.py b/packages/flow/openadapt_flow/__main__.py index bb52425ba8..9bc35c8fed 100644 --- a/packages/flow/openadapt_flow/__main__.py +++ b/packages/flow/openadapt_flow/__main__.py @@ -5230,6 +5230,87 @@ def _cmd_approver_grant(args: argparse.Namespace) -> int: return 0 +def _cmd_teach_authorization(args: argparse.Namespace) -> int: + """Create, sign or check a teach authorization (what a teaching agent may do).""" + import json + import sys + from datetime import datetime, timedelta, timezone + + from openadapt_flow import teach_authorization as ta + from openadapt_flow.approval_mode.human_attestation import ( + AttestationRefused, + load_signer, + ) + + def refuse(code: str, message: str) -> int: + print(json.dumps({"ok": False, "code": code, "message": message})) + return 2 + + if args.teach_auth_cmd == "create": + try: + spec = json.loads(Path(args.spec).read_text(encoding="utf-8")) + if not 0 < args.expires_in_hours <= 24 * 90: + return refuse( + "teach_auth_malformed", "--expires-in-hours is from 0 to 2160" + ) + expires = datetime.now(timezone.utc) + timedelta( + hours=args.expires_in_hours + ) + path = ta.create( + Path(args.out), + spec, + authorizer=args.authorizer, + authorizer_public_key=args.authorizer_public_key, + expires_at=expires, + ) + except (OSError, ValueError, ta.TeachAuthorizationError) as exc: + return refuse(getattr(exc, "code", "teach_auth_malformed"), str(exc)) + print(json.dumps({"ok": True, "statement": str(path)})) + return 0 + try: + key_dir, root = ta.pinned_trust( + Path(args.key_dir) if args.key_dir else None, args.approver_root + ) + except ta.TeachAuthorizationError as exc: + return refuse(exc.code, str(exc)) + if args.teach_auth_cmd == "verify": + result = ta.verify(Path(args.statement), key_dir=key_dir, root=root) + if not result.ok: + return refuse(str(result.reason), result.detail) + record = ta.record(result) + print(json.dumps({"ok": True, **record})) + return 0 + # sign: the person whose approver key file it is, at a terminal. + if not (sys.stdin.isatty() and sys.stdout.isatty()): + return refuse( + "teach_auth_unsigned", + "teach-authorization sign needs a person at an interactive terminal", + ) + try: + signer = load_signer( + Path(args.approver_key), key_dir, method="person_key", root=root + ) + except AttestationRefused as exc: + return refuse(exc.code, str(exc)) + preview = ta.verify(Path(args.statement), key_dir=key_dir, root=root) + if preview.statement is None: + return refuse(str(preview.reason), preview.detail) + print(ta.render(preview)) + reply = input("Sign this teach authorization? Type yes to sign: ") + if reply.strip().lower() != "yes": + print("Not signed.") + return 1 + try: + ta.sign(Path(args.statement), signer) + except (ta.TeachAuthorizationError, AttestationRefused) as exc: + return refuse(getattr(exc, "code", "teach_auth_unsigned"), str(exc)) + result = ta.verify(Path(args.statement), key_dir=key_dir, root=root) + if not result.ok: + return refuse(str(result.reason), result.detail) + print(json.dumps({"ok": True, **ta.record(result)})) + return 0 + + def _answer_durable_item_pause(run_dir: Path, request, answer) -> bool: """Carry an answer to a durable pause the run left after waiting. @@ -10003,6 +10084,51 @@ def build_parser() -> argparse.ArgumentParser: q.add_argument("--key-dir", default=None) q.set_defaults(func=_cmd_approver_grant) + p = sub.add_parser( + "teach-authorization", + help=( + "Create, sign or check a teach authorization: what a teaching agent " + "may do, signed by a person with their approver key" + ), + ) + ta_sub = p.add_subparsers(dest="teach_auth_cmd", required=True) + q = ta_sub.add_parser("create", help="Write the unsigned statement for a spec") + q.add_argument( + "spec", help="JSON: task, purposes, origins, inputs, writes, forbidden" + ) + q.add_argument("--out", required=True, help="Folder for teach-authorization.json") + q.add_argument("--authorizer", required=True, help="Your principal on the roster") + q.add_argument( + "--authorizer-public-key", required=True, help="Your approver key (64 hex)" + ) + q.add_argument( + "--expires-in-hours", + type=float, + default=24.0, + help="How long it may be used to teach (default 24, at most 2160)", + ) + q.set_defaults(func=_cmd_teach_authorization) + for verb, text in ( + ("sign", "Show what the statement allows, then sign it with your approver key"), + ("verify", "Check a signed statement against the approver roster"), + ): + q = ta_sub.add_parser(verb, help=text) + q.add_argument("statement", help="teach-authorization.json or its folder") + q.add_argument("--key-dir", default=None, help="Holds the approver roster") + q.add_argument( + "--approver-root", + default=None, + help="The approver root to pin (default OPENADAPT_APPROVER_ROOT)", + ) + if verb == "sign": + q.add_argument( + "--approver-key", + required=True, + help="Your encrypted approver key file (passphrase from " + "OPENADAPT_JIT_KEY_PASSPHRASE)", + ) + q.set_defaults(func=_cmd_teach_authorization) + p = sub.add_parser( "bench", help="Replay a bundle N times against MockMed and aggregate" ) diff --git a/packages/flow/openadapt_flow/agent/browser_teach.py b/packages/flow/openadapt_flow/agent/browser_teach.py index 998520bc6e..cd82cc16a9 100644 --- a/packages/flow/openadapt_flow/agent/browser_teach.py +++ b/packages/flow/openadapt_flow/agent/browser_teach.py @@ -63,6 +63,13 @@ "default_tools_dir", ] +#: Why the lane teaches and runs no writes: nothing here binds a person's +#: signed teach authorization to what the agent does. +TAUGHT_WRITE_REFUSED = ( + "taught writes are refused in browser teach: a write an agent teaches needs a " + "person's signed teach authorization, and this lane can't bind one. Teach " + "the write with openadapt-agent serve --teach --teach-authorization" +) PROGRAM_SCHEMA = "openadapt.browser-tool/v1" PROGRAM_FILE = "program.json" TAUGHT_PREFIX = "taught_" @@ -1324,6 +1331,11 @@ def _start(self, args: Mapping[str, Any]) -> dict[str, Any]: ) contract_name = args.get("contract") contract = None + if contract_name is not None: + # A taught write is an agent teaching a write with no person's + # signed teach authorization bound to it (this lane has no teach + # scope guard), so it is refused. docs/TEACH_AUTHORIZATION.md. + raise BridgeError(TAUGHT_WRITE_REFUSED) if contract_name is not None: contract = self.contracts.get(str(contract_name)) if contract is None: @@ -2060,5 +2072,10 @@ def _run_taught(self, tool: str, args: Mapping[str, Any]) -> dict[str, Any]: "model_calls": 0, } if (program.get("result") or {}).get("kind") == "write": - return self._run_write(tool, program, inputs) + # A write tool taught here carries no person's teach authorization. + return { + "outcome": "did_not_run", + "what_happened": f"{TAUGHT_WRITE_REFUSED}; nothing was written", + "model_calls": 0, + } return self._replay(program, inputs) diff --git a/packages/flow/openadapt_flow/agent/cli.py b/packages/flow/openadapt_flow/agent/cli.py index 2714fd7861..caf1d279a4 100644 --- a/packages/flow/openadapt_flow/agent/cli.py +++ b/packages/flow/openadapt_flow/agent/cli.py @@ -313,6 +313,17 @@ def build_parser() -> argparse.ArgumentParser: metavar="DIR", help="With --teach: the JIT key directory (default $OPENADAPT_JIT_KEY_DIR or ~/.openadapt/jit)", ) + p.add_argument( + "--teach-authorization", + default=None, + metavar="STATEMENT", + help=( + "With --teach: a person's signed teach authorization (openadapt-flow " + "teach-authorization). Run 1 is scoped by it; without one, drafts " + "are refused at admission. With --repair-access: it lets a calling " + "agent's fix be approved (it must cover repair). Its signer is checked under the deployment config's approvers section (--config) only." + ), + ) p.add_argument( "--teach-hold", action="store_true", @@ -1150,6 +1161,45 @@ def _teach_write_adapter(ref: Optional[str]) -> Any: return load_write_adapter(ref) +def _served_authority(args: argparse.Namespace) -> Any: + """``--teach-authorization`` for a server the agent runs, or ``None``. + + The roster's root, key directory and head come only from the deployment + config's ``approvers`` section (``--config``): the agent starts this + server, so the environment and flags it sets never count. Without that + pin the statement verifies nowhere. + """ + + path = getattr(args, "teach_authorization", None) + if not path: + return None + from openadapt_flow.teach_authorization import TeachAuthority + + approvers = _config_approvers(args) + return TeachAuthority( + Path(path).expanduser().resolve(), + key_dir=Path(approvers.key_dir) if approvers and approvers.key_dir else None, + approver_root=approvers.root_public_key if approvers else None, + config_only=True, + at_least=tuple(approvers.head) if approvers and approvers.head else None, + ) + + +def _teach_authority(args: argparse.Namespace, key_dir: Optional[Path]) -> Any: + """``--teach-authorization``, checked now so a bad one stops the server.""" + + from openadapt_flow.teach_authorization import TeachAuthorizationError + + authority = _served_authority(args) + if authority is None: + return None + try: + authority.verify(purpose="teach") + except TeachAuthorizationError as exc: + raise ValueError(f"--teach-authorization doesn't verify: {exc}") from None + return authority + + def _serve_verify(args: argparse.Namespace, serve) -> int: """Serve only the verify tools over local stdio.""" from openadapt_flow.agent.verify_tools import VerifyBridge @@ -1316,6 +1366,7 @@ def _serve_teach(args: argparse.Namespace, serve) -> int: ), allow_origins=tuple(args.teach_allow_origin), write_adapter=_teach_write_adapter(args.teach_write_adapter), + teach_authorization=_teach_authority(args, key_dir), ) except Exception as exc: # noqa: BLE001 - every setup problem is a refusal print(f"serve: {exc}", file=sys.stderr) @@ -1462,10 +1513,12 @@ def _repair_desk(args: argparse.Namespace, runs_dir: Path): else: policy = RepairAccessPolicy(args.repair_access) demonstrator = LazyBackendDemonstrator(url=args.url, headed=args.headed) + authority = _served_authority(args) return RepairDesk( Path(args.repair_dir) if args.repair_dir else Path(runs_dir) / ".repair", policy, demonstrator=demonstrator, + teach_authorization=authority, ) diff --git a/packages/flow/openadapt_flow/agent/repair.py b/packages/flow/openadapt_flow/agent/repair.py index ad4547fe9f..8e73f5bfd4 100644 --- a/packages/flow/openadapt_flow/agent/repair.py +++ b/packages/flow/openadapt_flow/agent/repair.py @@ -405,7 +405,15 @@ def __init__( demonstrator: Optional[Demonstrator] = None, scrubber_factory: Callable[[], Any] = default_scrubber, agent_id: str = "calling-agent", + teach_authorization: Any = None, ) -> None: + """``teach_authorization`` (a + :class:`~openadapt_flow.teach_authorization.TeachAuthority` whose + statement covers ``repair`` and names the program's workflow name as + its task) is + what lets a calling agent's fix be approved: without one, the fix is + still proposed, and its approval is refused.""" + self.teach_authorization = teach_authorization self.root = Path(root) self.policy = policy self.repair_store_root = Path(repair_store_root or self.root / "repair-store") @@ -707,6 +715,7 @@ def _teach( raise RepairError("x and y must be whole pixel coordinates") if not (0 <= x < width and 0 <= y < height): raise RepairError("the click is outside the screen") + authorized = self._authorize(record) # Use the token once, before anything acts, so a retry can't click twice. record["state"] = "used" _write_json(self._token_path(sha), record) @@ -737,6 +746,13 @@ def _teach( screens_scrubbed=bool(record.get("screens_scrubbed", True)), scrub_evidence=ScreenScrubEvidence(**evidence) if evidence else None, screen_share_approved_by=approval.get("approved_by") if approval else None, + teach_authorization=( + RepairStore( + self.store_for(record["workflow"]) + ).keep_teach_authorization(*authorized) + if authorized is not None + else None + ), ) try: result = agent_teach.teach_from_calling_agent( @@ -783,6 +799,32 @@ def _teach( ) return response + def _authorize(self, record: dict[str, Any]) -> Optional[tuple[Any, Path]]: + """The operator's teach authorization for this repair, verified now. + + ``None`` when the server has none (the fix can't be approved). One + that doesn't verify, doesn't cover ``repair`` or names another task + refuses before anything acts. + """ + authority = self.teach_authorization + if authority is None: + return None + from openadapt_flow import teach_authorization as ta + + try: + from openadapt_flow.ir import Workflow + + # The task is the program's own name, which approval checks too. + task_id = Workflow.load(Path(record["bundle_dir"])).name + verification = authority.verify(purpose="repair", task_id=task_id) + except (ta.TeachAuthorizationError, OSError, ValueError) as exc: + raise RepairError( + "the teach authorization doesn't allow this repair " + f"({getattr(exc, 'code', type(exc).__name__)}); " + "nothing was done" + ) from None + return verification, Path(authority.statement) + def _demos_path(self, record: dict[str, Any]) -> Path: key = _sha(f"{record['workflow']}\0{record['step_id']}")[:16] return self.root / "demos" / f"{key}.json" diff --git a/packages/flow/openadapt_flow/agent/teach.py b/packages/flow/openadapt_flow/agent/teach.py index bd26a4c4bd..3357a4bc9a 100644 --- a/packages/flow/openadapt_flow/agent/teach.py +++ b/packages/flow/openadapt_flow/agent/teach.py @@ -156,6 +156,11 @@ class TeachConfig: allow_origins: tuple[str, ...] = () #: The app's write adapter: run 1's writes are read before they are sent. write_adapter: Any = None + #: A person's signed teach authorization + #: (:class:`~openadapt_flow.teach_authorization.TeachAuthority`). Run 1 is + #: scoped by it and its task id is the authorized one; without it every + #: draft this server teaches is refused at admission. + teach_authorization: Any = None #: One writer at a time on the app: run 1, the canaries and taught runs #: would otherwise count each other's records in their checks. @@ -740,7 +745,8 @@ def _compiler_for(self, args: Mapping[str, Any]) -> Any: verify=self.config.verify, expect=record, on_screen=list(args["on_screen"]), - task_id=f"taught.{name}" if name else None, + task_id=self._authorized_task_id() + or (f"taught.{name}" if name else None), policy=self.config.policy, cache_dir=self.config.cache_dir, key_dir=self.config.key_dir, @@ -749,10 +755,30 @@ def _compiler_for(self, args: Mapping[str, Any]) -> Any: on_warning=lambda text: None, allow_origins=self.config.allow_origins, write_adapter=self.config.write_adapter, + teach_authorization=self.config.teach_authorization, ), inputs except ValueError as exc: raise BridgeError(str(exc)) from None + def _authorized_task_id(self) -> Optional[str]: + """The task the operator's teach authorization names, if there is one. + + Read from the statement only to name the task; run 1 verifies the + signature before the agent acts, and admission verifies it again. + """ + + authority = self.config.teach_authorization + if authority is None: + return None + from openadapt_flow import teach_authorization as ta + + try: + return ta.statement_task_id(authority.statement) + except ta.TeachAuthorizationError as exc: + raise BridgeError( + f"the teach authorization is unusable ({exc.code})" + ) from None + def _start(self, args: Mapping[str, Any]) -> dict[str, Any]: from openadapt_flow.compile_agent.driven import DrivenRun from openadapt_flow.jit.compile_agent import RECORDING_WARNING @@ -928,9 +954,12 @@ def _load(self, task_key: Any) -> Any: task_dir = self.config.cache_dir / task_key if not (task_dir / "compile_agent.json").is_file(): raise BridgeError("unknown task_key") - return FirstRunCompiler.load( + compiler = FirstRunCompiler.load( task_dir, reset=self.config.reset, open_backend=self.config.open_backend ) + # Admission checks the signature under the same roster pin. + compiler.teach_authorization = self.config.teach_authorization + return compiler def _status(self, task_key: Any) -> dict[str, Any]: compiler = self._load(task_key) diff --git a/packages/flow/openadapt_flow/compile_agent/cli.py b/packages/flow/openadapt_flow/compile_agent/cli.py index 45e7831d4c..bf70175cc8 100644 --- a/packages/flow/openadapt_flow/compile_agent/cli.py +++ b/packages/flow/openadapt_flow/compile_agent/cli.py @@ -53,6 +53,21 @@ def _write_adapter(ref: Optional[str]) -> Any: raise SystemExit(f"--write-adapter: {exc}") from None +def _teach_authority(args: argparse.Namespace) -> Any: + """``--teach-authorization`` as a TeachAuthority, checked now; or None.""" + + path = getattr(args, "teach_authorization", None) + if not path: + return None + from openadapt_flow.teach_authorization import TeachAuthority + + return TeachAuthority( + Path(path), + key_dir=Path(args.key_dir) if args.key_dir else None, + approver_root=getattr(args, "approver_root", None), + ) + + def _pairs(items: Optional[list[str]], flag: str) -> dict[str, str]: out: dict[str, str] = {} for item in items or []: @@ -146,6 +161,23 @@ def add_parser(sub: argparse._SubParsersAction) -> None: "run 1's writes are read before they are sent" ), ) + p.add_argument( + "--teach-authorization", + metavar="STATEMENT", + help=( + "a person's signed teach authorization (openadapt-flow " + "teach-authorization): run 1 is scoped by it, and the flags may only " + "narrow it. Without one the draft is refused at admission" + ), + ) + p.add_argument( + "--approver-root", + default=None, + help=( + "the approver root the teach authorization's signer is checked under " + "(default OPENADAPT_APPROVER_ROOT)" + ), + ) p.add_argument( "--agent", default="claude", @@ -398,6 +430,7 @@ def cmd_compile_agent( allow_origins=args.allow_origin, constants=args.allow_text, write_adapter=_write_adapter(args.write_adapter), + teach_authorization=_teach_authority(args), ) out = result.to_dict() if result.status == "draft" and not tty: @@ -426,6 +459,7 @@ def _cmd_admit( if not args.task_dir: raise SystemExit("compile-agent admit needs the compiled task directory") compiler = FirstRunCompiler.load(args.task_dir) + compiler.teach_authorization = _teach_authority(args) jit = compiler._jit() if jit.admitted: _print({"status": "admitted", "detail": "already admitted"}, args.json) diff --git a/packages/flow/openadapt_flow/compile_agent/pipeline.py b/packages/flow/openadapt_flow/compile_agent/pipeline.py index 198d99c72b..6c4b2f71d5 100644 --- a/packages/flow/openadapt_flow/compile_agent/pipeline.py +++ b/packages/flow/openadapt_flow/compile_agent/pipeline.py @@ -377,6 +377,7 @@ def __init__( allow_origins: Sequence[str] = (), constants: Sequence[str] = (), write_adapter: Optional[Any] = None, + teach_authorization: Optional[Any] = None, ) -> None: from openadapt_flow.record_sor import load_observer @@ -431,6 +432,11 @@ def __init__( self.allow_origins = [str(o) for o in allow_origins] self.constants = [str(c) for c in constants if str(c)] self.write_adapter = write_adapter + #: A person's signed teach authorization + #: (:class:`~openadapt_flow.teach_authorization.TeachAuthority`). Run 1 + #: is scoped by it (the flags may only narrow it); without one the + #: draft is refused at admission. + self.teach_authorization = teach_authorization self._baseline: Optional[list[dict[str, Any]]] = None self._pending: Optional[tuple[Any, dict[str, str], Path]] = None self._receipt_baseline: Optional[Path] = None @@ -480,7 +486,12 @@ def _jit(self, agent: Any = None) -> CompiledAgent: model_key=self.model_key, key_dir=self.key_dir, on_warning=self.on_warning, - teach_scope=self.teach_scope, + # A signed authorization is the whole scope: the person who + # signed it named the origins, record, fields and typeable text. + teach_scope=None + if self.teach_authorization is not None + else self.teach_scope, + teach_authorization=self.teach_authorization, ) def teach_scope(self, params: Mapping[str, Any]) -> Any: @@ -1311,6 +1322,7 @@ def compile_first_run( allow_origins: Sequence[str] = (), constants: Sequence[str] = (), write_adapter: Optional[Any] = None, + teach_authorization: Optional[Any] = None, ) -> CompileResult: """Turn an agent's first verified run of ``prompt`` into an admitted program. @@ -1349,6 +1361,9 @@ def compile_first_run( write_adapter: The app's :class:`~openadapt_flow.teach_scope.WriteAdapter`, so run 1's writes are read before they are sent. + teach_authorization: A person's signed teach authorization + (:class:`~openadapt_flow.teach_authorization.TeachAuthority`). Run + 1 is scoped by it; without one the draft is refused at admission. The agent is called once, only when the task has no draft and no admitted program. @@ -1372,6 +1387,7 @@ def compile_first_run( allow_origins=allow_origins, constants=constants, write_adapter=write_adapter, + teach_authorization=teach_authorization, ) jit = compiler._jit() first: Optional[CompileResult] = None diff --git a/packages/flow/openadapt_flow/integrations/agents/skills.py b/packages/flow/openadapt_flow/integrations/agents/skills.py index 50f7eb090e..dd6b76bf5a 100644 --- a/packages/flow/openadapt_flow/integrations/agents/skills.py +++ b/packages/flow/openadapt_flow/integrations/agents/skills.py @@ -352,17 +352,26 @@ def key(self, prompt: str, params: Mapping[str, Any]) -> dict[str, Any]: "environment": self.environment, } - def _compiler(self, prompt: str, params: Mapping[str, Any]) -> Any: - from openadapt_flow.compile_agent import FirstRunCompiler + def task_id(self, prompt: str, params: Mapping[str, Any]) -> str: + """The task id this prompt compiles under. + + A ``teach_authorization`` (passed through to the compiler) must name + exactly this task, or run 1 is refused before the agent acts. + """ key = self.key(prompt, params) digest = hashlib.sha256( json.dumps(key, sort_keys=True).encode("utf-8") ).hexdigest()[:16] - base = self.common.get("task_id") or "skill" + return f"{self.common.get('task_id') or 'skill'}-{digest}" + + def _compiler(self, prompt: str, params: Mapping[str, Any]) -> Any: + from openadapt_flow.compile_agent import FirstRunCompiler + + key = self.key(prompt, params) common = {k: v for k, v in self.common.items() if k not in ("reset", "task_id")} compiler = FirstRunCompiler( - prompt, params=list(params), task_id=f"{base}-{digest}", **common + prompt, params=list(params), task_id=self.task_id(prompt, params), **common ) return compiler, key diff --git a/packages/flow/openadapt_flow/jit/cli.py b/packages/flow/openadapt_flow/jit/cli.py index d0a0de2a1f..6ba773ea38 100644 --- a/packages/flow/openadapt_flow/jit/cli.py +++ b/packages/flow/openadapt_flow/jit/cli.py @@ -267,7 +267,10 @@ def confirm(prompt: str) -> str: authority.load_encrypted_key(args.key), _keys(args), policy=policy_doc ) decision = lifecycle.admit_with_key( - program, principal=principal, key_dir=_keys(args) + program, + principal=principal, + key_dir=_keys(args), + approver_root=getattr(args, "approver_root", None), ) if decision["outcome"] == "needs_person": print( @@ -293,6 +296,7 @@ def confirm(prompt: str) -> str: confirm=confirm(f"Admit as {args.principal}?"), key_dir=_key_dir(args), require_distinct_reviewer_and_admitter=args.require_distinct, + approver_root=getattr(args, "approver_root", None), ) print( f"ADMITTED by {manifest['admitted_by_principal_id']} at " @@ -403,6 +407,12 @@ def common(p: argparse.ArgumentParser, *, program: bool = True) -> None: action="store_true", help="Refuse if you also reviewed the draft", ) + p.add_argument( + "--approver-root", + default=None, + help="The approver root the draft's teach authorization is checked " + "under (default OPENADAPT_APPROVER_ROOT)", + ) p = verbs.add_parser("reject", help="Set the draft aside") common(p) p.add_argument("--principal", help="Your name or user id") diff --git a/packages/flow/openadapt_flow/jit/compile_agent.py b/packages/flow/openadapt_flow/jit/compile_agent.py index c5b8ff6c5d..1d45c90d08 100644 --- a/packages/flow/openadapt_flow/jit/compile_agent.py +++ b/packages/flow/openadapt_flow/jit/compile_agent.py @@ -537,7 +537,12 @@ def __enter__(self) -> "Recording": inputs=dict(self._params), model_key=owner.model_key, ) - scope = owner._teach_scope(self._params) + try: + scope = owner._teach_scope(self._params) + except Exception: + # An authorization that does not verify or fit leaves no draft. + shutil.rmtree(program, ignore_errors=True) + raise if scope is not None: from openadapt_flow.teach_scope import TeachGuard @@ -705,6 +710,11 @@ class CompiledAgent: on_warning: Optional[Callable[[str], None]] = None require_distinct_reviewer_and_admitter: bool = False teach_scope: Optional[TeachScopeSource] = None + #: A person's signed teach authorization + #: (:class:`~openadapt_flow.teach_authorization.TeachAuthority`). With + #: one, the recording's scope is bound from it; ``teach_scope`` may only + #: narrow it. Without one the draft can't be admitted. + teach_authorization: Any = None _sleep: Callable[[float], None] = field(default=time.sleep, repr=False) # -- layout -------------------------------------------------------------- @@ -752,7 +762,9 @@ def state(self) -> Optional[lifecycle.BundleState]: def status(self) -> dict[str, Any]: """The draft's steps and which admission requirements it meets.""" - return lifecycle.admission_status(self.program_dir, key_dir=self.key_dir) + return lifecycle.admission_status( + self.program_dir, key_dir=self.key_dir, trust=self.teach_authorization + ) # -- run ----------------------------------------------------------------- @@ -812,11 +824,28 @@ def _teach_scope(self, params: Mapping[str, Any]) -> Optional[TeachScope]: from openadapt_flow.teach_scope import TeachScope source = self.teach_scope - if source is None: + authority = self.teach_authorization + if source is None and authority is None: return None - scope = source if isinstance(source, TeachScope) else source(dict(params)) + authorized = None + if authority is not None: + from openadapt_flow import teach_authorization as ta + + # Verify the copy kept beside the draft, so it is the one admitted. + kept = authority.kept_in( + self.program_dir / lifecycle.TEACH_AUTHORIZATION_DIR + ) + authorized = kept.scope( + purpose="teach", task_id=self.task.task_id, params=params + ) + if source is None: + scope = authorized + else: + scope = source if isinstance(source, TeachScope) else source(dict(params)) if not isinstance(scope, TeachScope): raise ValueError("teach_scope must be a TeachScope or return one") + if authorized is not None and scope is not authorized: + scope = ta.narrowed(scope, authorized) if scope.task_id != self.task.task_id: raise ValueError( f"the teach scope is for task {scope.task_id!r}, not " @@ -921,6 +950,10 @@ def _write_draft( **guard.report(), "readback": readback.to_dict() if readback is not None else None, } + if guard.scope.authorization is not None: + # Who signed the scope; the statement is kept beside the + # draft (lifecycle.TEACH_AUTHORIZATION_DIR) for admission. + candidate["teach_authorization"] = dict(guard.scope.authorization) # Who made the recording, signed with the bundle digest, so editing # candidate.json can't turn the recorder into someone else. candidate = lifecycle.sign_recorder(program, candidate, key_dir=self.key_dir) @@ -1131,6 +1164,7 @@ def admit(self, *, principal_id: str, confirm: object = None) -> dict[str, Any]: require_distinct_reviewer_and_admitter=( self.require_distinct_reviewer_and_admitter ), + trust=self.teach_authorization, ) def admit_with_key(self, principal: Any) -> dict[str, Any]: @@ -1139,7 +1173,11 @@ def admit_with_key(self, principal: Any) -> dict[str, Any]: See :func:`openadapt_flow.jit.lifecycle.admit_with_key`.""" return lifecycle.admit_with_key( - self.program_dir, principal=principal, key_dir=self.key_dir + self.program_dir, + principal=principal, + key_dir=self.key_dir, + # The roster pin the teach authorization was checked under. + trust=self.teach_authorization, ) def reject(self, *, principal_id: Any, reason: str = "") -> Path: @@ -1379,6 +1417,7 @@ def compile_agent( on_warning: Optional[Callable[[str], None]] = None, require_distinct_reviewer_and_admitter: bool = False, teach_scope: Optional[TeachScopeSource] = None, + teach_authorization: Any = None, ) -> CompiledAgent: """Wrap ``agent`` so a task is explored once and then run as a program. @@ -1417,6 +1456,11 @@ def compile_agent( system of record is read back after the run. A violation ends the capture as ``scope_refused`` or ``readback_refused`` with no draft. See ``docs/THREAT_MODEL.md``. + teach_authorization: A person's signed teach authorization + (:class:`~openadapt_flow.teach_authorization.TeachAuthority`). + The recording's scope is bound from it, and ``teach_scope`` may + only narrow it. Admission refuses an agent-taught draft without + one. """ if agent is not None: @@ -1446,4 +1490,5 @@ def compile_agent( on_warning=on_warning, require_distinct_reviewer_and_admitter=require_distinct_reviewer_and_admitter, teach_scope=teach_scope, + teach_authorization=teach_authorization, ) diff --git a/packages/flow/openadapt_flow/jit/lifecycle.py b/packages/flow/openadapt_flow/jit/lifecycle.py index 7b5cc24b0b..a223d22bf7 100644 --- a/packages/flow/openadapt_flow/jit/lifecycle.py +++ b/packages/flow/openadapt_flow/jit/lifecycle.py @@ -105,7 +105,17 @@ DRAFT_ORIGIN = "agent_run" #: The fields of ``candidate.json`` that say who made the recording. They are #: signed with the bundle digest in the candidate's ``recorder`` block. -RECORDER_FIELDS = ("recorded_by", "agent_principal_id", "dialect") +RECORDER_FIELDS = ( + "recorded_by", + "agent_principal_id", + "dialect", + # What the agent was allowed to do and who signed it. Absent (None) on + # drafts from before teach authorizations, which still verify. + "teach_scope", + "teach_authorization", +) +#: Where a draft keeps a copy of the teach authorization it was taught under. +TEACH_AUTHORIZATION_DIR = "teach-authorization" EVENTS_FILE = "lifecycle.jsonl" ADMISSION_FILE = "admission.json" #: Every canary run, one JSON object per line. ``canary.json`` is the latest. @@ -761,6 +771,26 @@ def state_of( return None +def _teach_authorization_status( + program: Path, + candidate: Mapping[str, Any], + key_dir: Optional[Path], + trust: Any = None, +) -> dict[str, Any]: + try: + recorded = teach_authorization_gate( + program, candidate, key_dir=key_dir, trust=trust + ) + except AdmissionError as exc: + return {"met": False, "required": True, "problem": str(exc)} + return { + "met": True, + "required": True, + "sha256": recorded.get("sha256"), + "authorizer": recorded.get("authorizer"), + } + + def _evidence_at(path: Path, digest: str) -> Optional[dict[str, Any]]: record = _read(path) if record is None or record.get("digest") != digest: @@ -769,7 +799,7 @@ def _evidence_at(path: Path, digest: str) -> Optional[dict[str, Any]]: def admission_status( - program_dir: Path | str, *, key_dir: Optional[Path] = None + program_dir: Path | str, *, key_dir: Optional[Path] = None, trust: Any = None ) -> dict[str, Any]: """What the draft has and what it still needs, pinned to its digest.""" @@ -817,6 +847,9 @@ def admission_status( "met": bool(canary and canary.get("passed")), "outcome": (canary or {}).get("outcome"), }, + "teach_authorization": _teach_authorization_status( + program, candidate, key_dir, trust + ), }, "needs_person": _evidence_at(program / NEEDS_PERSON_FILE, digest), # A policy such as clinical-write needs one on every write; a person @@ -1207,6 +1240,8 @@ def admit( confirm: object = None, key_dir: Optional[Path] = None, require_distinct_reviewer_and_admitter: bool = False, + approver_root: Optional[str] = None, + trust: Any = None, ) -> dict[str, Any]: """A person admits a DRAFT and the signed admission manifest is written. @@ -1255,7 +1290,9 @@ def admit( f"{name!r} reviewed this draft; the policy requires a different " "principal to admit it" ) - canary = _evidence_gates(program, candidate, workflow, digest, key_dir) + canary = _evidence_gates( + program, candidate, workflow, digest, key_dir, approver_root, trust + ) assert canary is not None return _write_manifest( program, @@ -1276,6 +1313,8 @@ def _evidence_gates( workflow: Any, digest: str, key_dir: Optional[Path] = None, + approver_root: Optional[str] = None, + trust: Any = None, ) -> Optional[dict[str, Any]]: """The checks every admission mode makes: parameters confirmed, a passing canary for this digest, independent read-back on every write @@ -1284,6 +1323,9 @@ def _evidence_gates( from openadapt_flow.policy import evaluate_policy, load_policy + teach_authorization_gate( + program, candidate, key_dir=key_dir, approver_root=approver_root, trust=trust + ) task_params = _task_params(candidate) if task_params and _evidence_at(program / PARAMS_FILE, digest) is None: raise AdmissionError(f"confirm the parameters {task_params} first") @@ -1323,6 +1365,115 @@ def _evidence_gates( return canary +def teach_authorization_gate( + program: Path, + candidate: Mapping[str, Any], + *, + key_dir: Optional[Path] = None, + approver_root: Optional[str] = None, + trust: Any = None, +) -> dict[str, Any]: + """Every JIT draft needs a person's signed teach authorization. + + A JIT draft is compiled from an agent's run, so it is agent-taught. What + decides it is the signed recorder block (:func:`_recorder`), not an + editable field. The block's ``teach_authorization`` entry must name the + statement kept beside the draft (:data:`TEACH_AUTHORIZATION_DIR`), and + that statement must verify against the approver roster as it is now + (:mod:`openadapt_flow.teach_authorization`): the key directory's roster + under the pinned approver root (``approver_root``, else + ``OPENADAPT_APPROVER_ROOT``). An unknown, removed or wrong key, an + unpinned root, a changed statement, another task, or the recording agent + as signer refuses. The capture must also have run under the scope the + authorization binds, with a verified read-back. Returns the recorded + entry. ``trust`` (a + :class:`~openadapt_flow.teach_authorization.TeachAuthority`) gives the + roster's key directory, root and recorded heads instead, as the process + that taught it pinned them (the deployment config's, for a server an + agent runs). + + Drafts from a path that can't bind an authorization (the airbag + handoff, ``agent_guard.handoff``) record none, so they are refused here. + """ + + from openadapt_flow import teach_authorization as ta + + keys = Path(key_dir) if key_dir is not None else default_key_dir() + recorder = _recorder(program, keys) + recorded = recorder.get("teach_authorization") + if not recorded: + dialect = str(recorder.get("dialect") or "") + why = ( + " Drafts from the airbag handoff can't carry one yet; teach the task " + "with a teach authorization instead." + if dialect == "attached-browser" + else "" + ) + raise AdmissionError( + "an agent-taught draft needs a person's signed teach authorization " + f"(teach_auth_not_recorded).{why}" + ) + statement = program / TEACH_AUTHORIZATION_DIR / ta.STATEMENT_FILE + scope = recorder.get("teach_scope") or {} + try: + if trust is not None: + trust_dir, root = trust.trust() + else: + trust_dir, root = ta.pinned_trust(keys, approver_root) + ta.check_admission( + recorded, + statement, + key_dir=trust_dir, + root=root, + # The task the signed recorder block names. + task_id=str((scope.get("scope") or {}).get("task_id") or "") or None, + not_signed_by=recorder.get("agent_principal_id"), + at_least=getattr(trust, "at_least", None), + ) + except ta.TeachAuthorizationError as exc: + raise AdmissionError( + f"an agent-taught draft needs a person's signed teach authorization " + f"({exc.code}): {exc}" + ) from exc + if (scope.get("scope") or {}).get("task_id") != (candidate.get("task") or {}).get( + "task_id" + ): + raise AdmissionError( + "the draft's task is not the one its capture was scoped to " + "(teach_auth_task_mismatch)" + ) + bound = (scope.get("scope") or {}).get("authorization") or {} + readback = scope.get("readback") or {} + if bound.get("sha256") != recorded.get("sha256"): + raise AdmissionError( + "the draft was not captured under the scope its teach authorization " + "binds (teach_auth_not_bound)" + ) + if readback.get("status") != "verified": + raise AdmissionError( + "the authorized capture's read-back did not verify " + "(teach_auth_readback_unverified)" + ) + return dict(recorded) + + +def _keep_authorization(source: Path, bundle: Path) -> None: + """Keep the teach authorization's statement inside the admitted bundle. + + A repaired copy carries it from there (:func:`admit_repair`), so its + signature can be checked again wherever the bundle goes. + """ + + from openadapt_flow import teach_authorization as ta + + statement = source / ta.STATEMENT_FILE + if statement.is_file(): + try: + ta.copy_into(statement, bundle / TEACH_AUTHORIZATION_DIR) + except ta.TeachAuthorizationError as exc: + raise AdmissionError(str(exc)) from exc + + def _write_manifest( program: Path, candidate: Mapping[str, Any], @@ -1383,6 +1534,11 @@ def _write_manifest( "signing_key": fingerprint_of(private.public_key()), **(extra or {}), } + if candidate.get("teach_authorization"): + # The bundle carries who authorized the teaching. Only written when + # present, so manifests from before teach authorizations verify. + body["teach_authorization"] = candidate["teach_authorization"] + _keep_authorization(program / TEACH_AUTHORIZATION_DIR, program / "bundle") if mode == "agent_policy" and _is_recording_agent(name, recorder): # Only written when true, so manifests from before self-admission # existed verify unchanged. @@ -1412,6 +1568,8 @@ def admit_with_key( principal: Any, key_dir: Optional[Path] = None, now: Optional[Any] = None, + approver_root: Optional[str] = None, + trust: Any = None, ) -> dict[str, Any]: """Admit without a person at a terminal. Returns ``{"outcome": ...}``. @@ -1452,7 +1610,9 @@ def admit_with_key( workflow = _load_workflow(program / "bundle") digest = _digest(workflow) if principal.kind == "admitter_key": - canary = _evidence_gates(program, candidate, workflow, digest, keys) + canary = _evidence_gates( + program, candidate, workflow, digest, keys, approver_root, trust + ) assert canary is not None manifest = _write_manifest( program, @@ -1521,7 +1681,9 @@ def admit_with_key( f"{len(clean)} clean canaries for this bundle; the policy needs " f"{policy.min_clean_canaries}" ) - canary = _evidence_gates(program, candidate, workflow, digest, keys) + canary = _evidence_gates( + program, candidate, workflow, digest, keys, approver_root, trust + ) assert canary is not None manifest = _write_manifest( program, @@ -1678,6 +1840,7 @@ def admit_repair( route: Optional[str] = None, probe_only: bool = False, key_dir: Optional[Path] = None, + approver_root: Optional[str] = None, ) -> dict[str, Any]: """Carry an admitted program's admission to a repaired copy of it. @@ -1694,9 +1857,16 @@ def admit_repair( approved the repair, or ``standard-change:`` for a fix a signed repair policy allowed. + The repair is taught too (a model re-anchors steps or demonstrates a + bridge), so the prior admission's teach authorization must cover + ``repair`` and still verify against the approver roster + (:func:`_repair_authorization`); its statement is copied into the + repaired bundle. + Raises: - AdmissionError: the prior program is not admitted, the copy changes - its record check or parameters, or no approver is named. + AdmissionError: the prior program is not admitted, has no teach + authorization that covers repair, the copy changes its record + check or parameters, or no approver is named. """ from openadapt_flow.execute.keys import ( @@ -1710,6 +1880,9 @@ def admit_repair( prior = verify_admission(prior_bundle, key_dir=keys) if prior.get("admission_mode") == "repair_probe": raise AdmissionError("a repair probe's admission can't carry to another copy") + _repair_authorization( + prior, Path(prior_bundle), Path(repaired_bundle), keys, approver_root + ) prior_workflow = _load_workflow(Path(prior_bundle)) repaired = _load_workflow(Path(repaired_bundle)) if sorted(repaired.params) != sorted(prior.get("params") or []): @@ -1753,6 +1926,34 @@ def admit_repair( return manifest +def _repair_authorization( + prior: Mapping[str, Any], + prior_bundle: Path, + repaired_bundle: Path, + keys: Path, + approver_root: Optional[str], +) -> None: + from openadapt_flow import teach_authorization as ta + + recorded = prior.get("teach_authorization") + try: + trust_dir, root = ta.pinned_trust(keys, approver_root) + ta.check_admission( + recorded, + prior_bundle / TEACH_AUTHORIZATION_DIR / ta.STATEMENT_FILE, + key_dir=trust_dir, + root=root, + purpose="repair", + not_signed_by=prior.get("agent_principal_id"), + ) + except ta.TeachAuthorizationError as exc: + raise AdmissionError( + "a repair of an agent-taught program needs its teach authorization " + f"to cover repair ({exc.code}): {exc}" + ) from exc + _keep_authorization(prior_bundle / TEACH_AUTHORIZATION_DIR, repaired_bundle) + + def _verify_repair(body: Mapping[str, Any], keys: Path) -> None: """A repaired copy's admission: the prior one verifies, and nothing that was admitted changed except the anchors and steps the repair touched.""" diff --git a/packages/flow/openadapt_flow/mockmed/teach_scope.py b/packages/flow/openadapt_flow/mockmed/teach_scope.py index 170bc0b60c..a667e1d667 100644 --- a/packages/flow/openadapt_flow/mockmed/teach_scope.py +++ b/packages/flow/openadapt_flow/mockmed/teach_scope.py @@ -16,7 +16,7 @@ from openadapt_flow.teach_scope import TeachScope, WriteClaim -__all__ = ["EncounterWrites", "encounter_scope"] +__all__ = ["EncounterWrites", "encounter_authorization_spec", "encounter_scope"] class EncounterWrites: @@ -70,3 +70,53 @@ def encounter_scope( constants=constants, write_adapter=EncounterWrites(), ) + + +def encounter_authorization_spec( + base_url: str, + *, + patient_id: str, + note: Optional[str] = None, + note_sha256: Optional[str] = None, + encounter_type: str = "Triage", + purposes: tuple[str, ...] = ("teach", "repair"), + forbidden_controls: tuple[str, ...] = ("Delete encounter",), +) -> dict[str, Any]: + """The teach authorization a person signs for MockMed's add-a-note task. + + Give the note as ``note`` (spelled out in the statement) or as + ``note_sha256`` (the operator supplies the value at teach time). + """ + + from openadapt_flow.teach_scope import origin_of + + if (note is None) == (note_sha256 is None): + raise ValueError("give note or note_sha256") + return { + "task": { + "task_id": "mockmed.add_note", + "summary": "Add one triage encounter note to the named patient", + }, + "purposes": list(purposes), + "origins": [origin_of(base_url)], + "inputs": { + "note": {"value": note} if note is not None else {"sha256": note_sha256} + }, + "writes": { + "record_kind": "encounter", + "identity": {"patient_id": {"value": patient_id}}, + "fields": {"type": {"value": encounter_type}, "note": {"input": "note"}}, + "expected_rows": 1, + "write_adapter": "openadapt_flow.mockmed.teach_scope:EncounterWrites", + }, + "forbidden": { + "actions": [ + "drag", + "delete_request", + "new_window", + "download", + "file_upload", + ], + "controls": list(forbidden_controls), + }, + } diff --git a/packages/flow/openadapt_flow/repair/candidate.py b/packages/flow/openadapt_flow/repair/candidate.py index 0de17ac83b..17e19fd05a 100644 --- a/packages/flow/openadapt_flow/repair/candidate.py +++ b/packages/flow/openadapt_flow/repair/candidate.py @@ -273,6 +273,11 @@ class RepairProvenance(BaseModel): #: The person who approved sharing for this one repair token #: (``ask_each_time`` only). screen_share_approved_by: Optional[str] = Field(default=None, max_length=256) + #: The person-signed teach authorization (purpose ``repair``) the fix was + #: demonstrated under (:func:`openadapt_flow.teach_authorization.record`). + #: Its statement is kept in the repair store; approval of a calling + #: agent's fix verifies it again and refuses without one. + teach_authorization: Optional[dict[str, Any]] = None @model_validator(mode="after") def _consistent(self) -> "RepairProvenance": diff --git a/packages/flow/openadapt_flow/repair/lifecycle.py b/packages/flow/openadapt_flow/repair/lifecycle.py index 0499de913b..7bb00dfcef 100644 --- a/packages/flow/openadapt_flow/repair/lifecycle.py +++ b/packages/flow/openadapt_flow/repair/lifecycle.py @@ -32,11 +32,12 @@ from __future__ import annotations import os +import re import shutil import tempfile from datetime import datetime from pathlib import Path -from typing import TYPE_CHECKING, Literal, Optional +from typing import TYPE_CHECKING, Any, Literal, Optional from pydantic import BaseModel, ConfigDict, Field @@ -637,6 +638,64 @@ def _require_digest(workflow: Workflow, expected: str, label: str) -> None: f"found {actual[:16] if actual else ''}...)" ) + def teach_authorization_dir(self, sha256: str) -> Path: + """Where the store keeps a teach authorization's statement, by digest.""" + + if not re.fullmatch(r"[0-9a-f]{64}", str(sha256)): + raise RepairLifecycleError("refused: not a statement digest") + return self.root / "teach-authorizations" / sha256 + + def keep_teach_authorization( + self, verification: Any, statement: Path + ) -> dict[str, Any]: + """Keep a verified teach authorization's statement; return its record.""" + + from openadapt_flow import teach_authorization as ta + + record = ta.record(verification) + ta.copy_into(statement, self.teach_authorization_dir(record["sha256"])) + return record + + def _require_teach_authorization(self, candidate: RepairCandidate) -> None: + """A fix a calling agent demonstrated needs a person's teach authorization. + + The provenance names the authorization (purpose ``repair``) the fix + was taught under; it must name this workflow as its task, and its + statement, kept in this store, must verify + against the approver roster as it is now (``$OPENADAPT_JIT_KEY_DIR`` + and ``OPENADAPT_APPROVER_ROOT``), and the calling agent may not be the + signer. Without one the approval is refused. + """ + + from openadapt_flow import teach_authorization as ta + + provenance = candidate.provenance + if provenance is None or provenance.repaired_by != "calling_agent": + return + recorded = provenance.teach_authorization + try: + key_dir, root = ta.pinned_trust() + sha = str((recorded or {}).get("sha256") or "") + statement = ( + self.teach_authorization_dir(sha) / ta.STATEMENT_FILE + if re.fullmatch(r"[0-9a-f]{64}", sha) + else None + ) + ta.check_admission( + recorded, + statement, + key_dir=key_dir, + root=root, + purpose="repair", + task_id=candidate.workflow_name, + not_signed_by=provenance.agent_id, + ) + except ta.TeachAuthorizationError as exc: + raise RepairLifecycleError( + "refused: a fix a calling agent demonstrated needs a person's " + f"signed teach authorization that covers repair ({exc.code}): {exc}" + ) from exc + def approve( self, candidate_id: str, @@ -667,6 +726,7 @@ def approve( "replay and fault campaigns" ) self._reverify_invariants(candidate) + self._require_teach_authorization(candidate) candidate.approval = ApprovalRecord( approved_by=actor.id, non_interactive=non_interactive, diff --git a/packages/flow/openadapt_flow/sealed_set.py b/packages/flow/openadapt_flow/sealed_set.py index 07c1c0f9ce..c2b7d05603 100644 --- a/packages/flow/openadapt_flow/sealed_set.py +++ b/packages/flow/openadapt_flow/sealed_set.py @@ -175,6 +175,11 @@ class FileKind: "a signed admission manifest, which can carry read-back values", ), FileKind("jit_draft.json", "phi", "draft state of a just-in-time program"), + FileKind( + "teach-authorization/*", + "phi", + "the person-signed teach authorization the program was taught under", + ), FileKind("admit.json", "phi", "authoring admission with the local login name"), FileKind("openadapt-card.json", "phi", "operator-written name, purpose and inputs"), FileKind( diff --git a/packages/flow/openadapt_flow/teach_authorization.py b/packages/flow/openadapt_flow/teach_authorization.py new file mode 100644 index 0000000000..d2890208d4 --- /dev/null +++ b/packages/flow/openadapt_flow/teach_authorization.py @@ -0,0 +1,1008 @@ +"""Teach authorizations: a person signs what a teaching agent may do. + +A teaching or repairing agent reads untrusted screen content, and what it +does can become a reusable program. :class:`~openadapt_flow.teach_scope.TeachScope` +limits its actions, but a scope the agent (or its harness) writes is only as +trustworthy as the agent. A teach authorization +(``openadapt.teach-authorization/v1``) moves the scope to a person: a short +JSON statement that names + +* the **task** (``task_id`` and a one-line summary); +* the **inputs** the agent may type, each as a value or as the SHA-256 of + the value the operator supplies at teach time (so the statement need not + hold record data); +* the app **origins** the browser may reach; +* the permitted **writes**: the record kind, the identity rule for the one + intended record (fields that must match exactly), the fields the save may + write and their values, how many rows, and the app's write adapter; +* the **forbidden** actions (a closed list) and controls (by label); +* what it covers (``teach``, ``repair`` or both) and when it expires. + +**Who signs.** A person on the signed approver roster +(:mod:`openadapt_flow.approval_mode.approver_roster`), with the same +approver key that answers saves. The signature is a +:class:`~openadapt_flow.approval_mode.human_attestation.HumanAttestation` +over :data:`DOMAIN` and the canonical statement, kept beside it in +``teach-authorization.json.sig``. Verification is the approver verifier's +own (:func:`~openadapt_flow.approval_mode.human_attestation.refusal`): the +signer must be on the roster the caller's pinned approver root vouches for, +and the roster must extend every head recorded earlier. So a key that was +never added, or was removed since (revoked), refuses, and so does a key +that holds another role or a principal that names an agent or model. + +Keys: a person's encrypted approver key file (``openadapt-flow jit keygen``, +added to the roster with ``openadapt-flow jit approver add``), signed at a +terminal with ``openadapt-flow teach-authorization sign``. Desktop's Touch ID +key answers only item approvals, so it doesn't sign these. The attestation's +``method`` label is not itself signed; what keeps another approval's +signature from counting here is the separate signing domain (:data:`DOMAIN`). + +Each authoring path that can be bound builds its scope with :func:`bind` +from a verified authorization, never from the agent's own description; the +paths that can't be bound are refused (docs/TEACH_AUTHORIZATION.md). A +draft records the authorization's digest (:func:`record`) and keeps a copy +of the statement; admission verifies that copy again against the roster as +it is then (:func:`check_admission`). +""" + +from __future__ import annotations + +import dataclasses +import hashlib +import json +import os +import re +import secrets +from dataclasses import dataclass, field +from datetime import datetime, timezone +from pathlib import Path +from typing import Any, Final, Literal, Mapping, Optional + +from pydantic import ( + BaseModel, + ConfigDict, + Field, + StrictInt, + StrictStr, + field_validator, + model_validator, +) + +SCHEMA: Final = "openadapt.teach-authorization/v1" +#: What an approver key signs for a teach authorization: this domain, then +#: the canonical statement (the approver verifier's message format). +DOMAIN: Final = b"openadapt.teach-authorization/v1\x00" +STATEMENT_FILE: Final = "teach-authorization.json" +SIGNATURE_FILE: Final = STATEMENT_FILE + ".sig" +#: The draft and bundle key that records the authorization. +RECORD_KEY: Final = "teach_authorization" +#: Where a draft, a bundle or a taught tool keeps its copy of the statement. +KEPT_DIR: Final = "teach-authorization" +#: The approver signing methods a teach authorization accepts. +SIGNING_METHODS: Final = ("desktop_key", "person_key") +#: Domain prefix for a value digest. +_VALUE_DOMAIN: Final = b"openadapt.teach-value/v1\0" + +_HEX64 = r"^[a-f0-9]{64}$" +_NAME = r"^[A-Za-z_][A-Za-z0-9_.-]{0,63}$" +_TASK_ID = r"^[A-Za-z0-9][A-Za-z0-9_.:-]{0,127}$" +_PRINCIPAL = r"^[A-Za-z0-9][A-Za-z0-9._:@-]{0,127}$" +_ISO_SECONDS = r"^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z$" +_ADAPTER = r"^[A-Za-z_][A-Za-z0-9_.]*:[A-Za-z_][A-Za-z0-9_]*$" + +#: Actions an authorization can forbid. Paste is always refused by the guard. +FORBIDDEN_ACTIONS: Final = ( + "paste", + "drag", + "right_click", + "delete_request", + "new_window", + "download", + "file_upload", +) +ForbiddenAction = Literal[ + "paste", + "drag", + "right_click", + "delete_request", + "new_window", + "download", + "file_upload", +] +Purpose = Literal["teach", "repair"] + +#: Closed refusal codes. None of them carries a value. +TeachAuthorizationRefusal = Literal[ + "teach_auth_missing", + "teach_auth_malformed", + "teach_auth_unsigned", + "teach_auth_bad_signature", + "teach_auth_signer_mismatch", + "teach_auth_unknown_signer", + "teach_auth_identity_unverified", + "teach_auth_method", + "teach_auth_agent_signer", + "teach_auth_expired", + "teach_auth_not_yet_valid", + "teach_auth_purpose", + "teach_auth_task_mismatch", + "teach_auth_input_missing", + "teach_auth_input_mismatch", + "teach_auth_input_not_declared", + "teach_auth_scope_widened", + "teach_auth_adapter_unloadable", + "teach_auth_not_recorded", + "teach_auth_digest_mismatch", +] + + +class TeachAuthorizationError(RuntimeError): + """A closed ``code`` and a detail without record values.""" + + def __init__(self, code: str, detail: str = "") -> None: + super().__init__(f"{code}: {detail}" if detail else code) + self.code = code + + +def value_sha256(value: str) -> str: + """The digest an authorization names for a value it does not spell out.""" + + return hashlib.sha256(_VALUE_DOMAIN + str(value).encode("utf-8")).hexdigest() + + +# -- the statement -------------------------------------------------------------- + + +class Bound(BaseModel): + """One permitted value: spelled out, by digest, or a declared input.""" + + model_config = ConfigDict(extra="forbid", frozen=True) + + value: Optional[StrictStr] = Field(default=None, min_length=1, max_length=4000) + sha256: Optional[StrictStr] = Field(default=None, pattern=_HEX64) + input: Optional[StrictStr] = Field(default=None, pattern=_NAME) + + @model_validator(mode="after") + def _one(self) -> "Bound": + given = [x for x in (self.value, self.sha256, self.input) if x is not None] + if len(given) != 1: + raise ValueError("a bound value is exactly one of value, sha256, input") + return self + + +class Task(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True) + + task_id: StrictStr = Field(pattern=_TASK_ID) + summary: StrictStr = Field(min_length=1, max_length=500) + + +class Writes(BaseModel): + """The one write the task may make.""" + + model_config = ConfigDict(extra="forbid", frozen=True) + + record_kind: StrictStr = Field(pattern=_NAME) + #: The identity rule: these fields of the written row must equal these + #: values (after whitespace and case normalization), and nothing else is + #: the intended record. + identity: dict[StrictStr, Bound] = Field(min_length=1, max_length=16) + #: The fields the save may write and the value each must have. + fields: dict[StrictStr, Bound] = Field(default_factory=dict, max_length=64) + expected_rows: StrictInt = Field(default=1, ge=1, le=100) + write_adapter: Optional[StrictStr] = Field(default=None, pattern=_ADAPTER) + allowed_write_paths: list[StrictStr] = Field(default_factory=list, max_length=32) + + @field_validator("identity", "fields") + @classmethod + def _names(cls, value: dict[str, Bound]) -> dict[str, Bound]: + for name in value: + if not re.match(_NAME, name): + raise ValueError(f"not a field name: {name!r}") + return value + + @field_validator("allowed_write_paths") + @classmethod + def _paths(cls, value: list[str]) -> list[str]: + for path in value: + if not path.startswith("/") or "?" in path or "#" in path: + raise ValueError(f"a write path is an exact URL path: {path!r}") + return value + + @model_validator(mode="after") + def _disjoint(self) -> "Writes": + overlap = set(self.identity) & set(self.fields) + if overlap: + raise ValueError("a name is both identity and field: " + ", ".join(overlap)) + return self + + +class Forbidden(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True) + + actions: list[ForbiddenAction] = Field(default_factory=list) + #: Labels of controls the agent may never click (exact, case and + #: whitespace normalized): "Delete encounter", "E-sign". + controls: list[StrictStr] = Field(default_factory=list, max_length=64) + + +class TeachAuthorizationV1(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True, populate_by_name=True) + + schema_: Literal["openadapt.teach-authorization/v1"] = Field( + default=SCHEMA, alias="schema" + ) + task: Task + purposes: list[Purpose] = Field(min_length=1, max_length=2) + origins: list[StrictStr] = Field(min_length=1, max_length=16) + inputs: dict[StrictStr, Bound] = Field(default_factory=dict, max_length=64) + #: Other text the agent may type (a test account's user name). + constants: list[Bound] = Field(default_factory=list, max_length=64) + writes: Writes + forbidden: Forbidden = Field(default_factory=Forbidden) + #: The approver who signs: their principal id on the approver roster. + authorizer: StrictStr = Field(pattern=_PRINCIPAL) + #: Their approver public key (64 hex). The signature must be this key's, + #: and the roster must hold this key for ``authorizer`` when it is checked. + authorizer_public_key: StrictStr = Field(pattern=_HEX64) + issued_at: StrictStr = Field(pattern=_ISO_SECONDS) + expires_at: Optional[StrictStr] = Field(default=None, pattern=_ISO_SECONDS) + nonce: StrictStr = Field(pattern=_HEX64) + + @field_validator("origins") + @classmethod + def _origins(cls, value: list[str]) -> list[str]: + from openadapt_flow.teach_scope import origin_of + + out = [] + for raw in value: + origin = origin_of(raw) + if origin is None or origin != raw.rstrip("/"): + raise ValueError(f"an origin is scheme://host[:port]: {raw!r}") + out.append(origin) + if len(set(out)) != len(out): + raise ValueError("origins are unique") + return out + + @field_validator("inputs") + @classmethod + def _inputs(cls, value: dict[str, Bound]) -> dict[str, Bound]: + for name, bound in value.items(): + if not re.match(_NAME, name): + raise ValueError(f"not an input name: {name!r}") + if bound.input is not None: + raise ValueError("an input is a value or a digest, not another input") + return value + + @model_validator(mode="after") + def _references(self) -> "TeachAuthorizationV1": + for where, bound in [ + *self.writes.identity.items(), + *self.writes.fields.items(), + *(("constant", c) for c in self.constants), + ]: + if bound.input is not None and bound.input not in self.inputs: + raise ValueError(f"{where} names undeclared input {bound.input!r}") + if len(set(self.purposes)) != len(self.purposes): + raise ValueError("purposes are unique") + if any(c.sha256 is not None for c in self.constants): + raise ValueError("a constant is a value or an input, not a digest") + return self + + def summary_line(self) -> str: + """What this authorization allows, without values.""" + + w = self.writes + return ( + f"{self.task.task_id}: one {w.record_kind} write " + f"(identity {', '.join(sorted(w.identity))}; fields " + f"{', '.join(sorted(w.fields)) or 'none'}) on " + f"{', '.join(self.origins)}; authorized by {self.authorizer}" + ) + + +@dataclass(frozen=True) +class Verification: + ok: bool + reason: Optional[str] = None + statement: Optional[TeachAuthorizationV1] = None + sha256: Optional[str] = None # of the statement's exact bytes + signature_sha256: Optional[str] = None + #: The roster principal whose approver key signed, once verified. + signer: Optional[str] = None + signer_public_key: Optional[str] = None + method: Optional[str] = None + #: The approver root the roster was checked under. + approver_root: Optional[str] = None + detail: str = field(default="") + + +def _paths(target: Path) -> tuple[Path, Path]: + target = Path(target) + statement = target / STATEMENT_FILE if target.is_dir() else target + return statement, statement.with_name(statement.name + ".sig") + + +def _utc_seconds(when: Optional[datetime]) -> str: + when = (when or datetime.now(timezone.utc)).astimezone(timezone.utc) + return when.strftime("%Y-%m-%dT%H:%M:%SZ") + + +def dumps(statement: TeachAuthorizationV1) -> str: + return ( + json.dumps( + statement.model_dump(by_alias=True, exclude_none=True), + indent=2, + sort_keys=True, + ) + + "\n" + ) + + +def create( + out_dir: Path, + spec: Mapping[str, Any], + *, + authorizer: str, + authorizer_public_key: str, + now: Optional[datetime] = None, + expires_at: Optional[datetime] = None, +) -> Path: + """Write the unsigned statement for ``spec`` in ``out_dir``; return its path. + + ``spec`` holds ``task``, ``purposes``, ``origins``, ``inputs``, + ``constants``, ``writes`` and ``forbidden`` as in the statement. + ``authorizer`` is the approver principal who will sign and + ``authorizer_public_key`` their approver key (64 hex). Refuses over an + existing statement. ``expires_at`` defaults to 24 hours from now and is + at most 90 days on. + """ + + path = Path(out_dir) / STATEMENT_FILE + if path.exists() or path.with_name(SIGNATURE_FILE).exists(): + raise TeachAuthorizationError("teach_auth_exists", f"{path} already exists") + from datetime import timedelta + + issued = (now or datetime.now(timezone.utc)).astimezone(timezone.utc) + expires_at = expires_at or issued + timedelta(hours=24) + if not issued < expires_at <= issued + timedelta(days=90): + raise TeachAuthorizationError( + "teach_auth_malformed", "expires_at is after now and at most 90 days on" + ) + statement = TeachAuthorizationV1.model_validate( + { + **dict(spec), + "schema": SCHEMA, + "authorizer": authorizer, + "authorizer_public_key": str(authorizer_public_key).lower(), + "issued_at": _utc_seconds(now), + "expires_at": _utc_seconds(expires_at), + "nonce": secrets.token_hex(32), + } + ) + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(dumps(statement), encoding="utf-8") + return path + + +def _load(statement_path: Path) -> tuple[bytes, dict[str, Any], TeachAuthorizationV1]: + from openadapt_flow.offline_seal import loads_strict + + try: + raw = Path(statement_path).read_bytes() + except FileNotFoundError as exc: + raise TeachAuthorizationError( + "teach_auth_missing", str(statement_path) + ) from exc + except OSError as exc: + raise TeachAuthorizationError( + "teach_auth_malformed", type(exc).__name__ + ) from exc + try: + data = loads_strict(raw) + statement = TeachAuthorizationV1.model_validate(data) + except Exception as exc: # noqa: BLE001 - any parse error is malformed + raise TeachAuthorizationError( + "teach_auth_malformed", type(exc).__name__ + ) from exc + if not isinstance(data, dict): + raise TeachAuthorizationError("teach_auth_malformed", "not an object") + return raw, data, statement + + +def statement_task_id(target: Path) -> str: + """The task a statement names, unverified (for naming the task only).""" + + statement_path, _sig = _paths(Path(target)) + return _load(statement_path)[2].task.task_id + + +def sign(statement_path: Path, signer: Any) -> Path: + """Sign the statement with an approver key; return the signature's path. + + ``signer`` is an + :class:`~openadapt_flow.approval_mode.human_attestation.ApproverSigner` + (``load_signer`` for a key file, ``stdio_signer`` for Desktop's key), + which names its principal from the roster. It must be the statement's + ``authorizer``, sign with ``authorizer_public_key``, and use a method in + :data:`SIGNING_METHODS`. Writes nothing on any mismatch. + """ + + from openadapt_flow.jit.authority import _verify + + statement_path, sig_path = _paths(Path(statement_path)) + _raw, data, statement = _load(statement_path) + if sig_path.exists(): + raise TeachAuthorizationError("teach_auth_already_signed", str(sig_path)) + if signer.method not in SIGNING_METHODS: + raise TeachAuthorizationError( + "teach_auth_method", f"{signer.method} can't sign a teach authorization" + ) + if signer.principal_id != statement.authorizer: + raise TeachAuthorizationError( + "teach_auth_signer_mismatch", + f"the statement names {statement.authorizer}; this key is " + f"{signer.principal_id} on the roster", + ) + attestation = signer.attest(DOMAIN, data) + if not _verify( + statement.authorizer_public_key, DOMAIN, data, attestation.signature + ): + raise TeachAuthorizationError( + "teach_auth_signer_mismatch", + "the signature isn't by the statement's authorizer_public_key", + ) + fd = os.open(sig_path, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o644) + with os.fdopen(fd, "w", encoding="utf-8") as handle: + handle.write(attestation.model_dump_json() + "\n") + return sig_path + + +_ROSTER_REFUSALS = { + "human_answer_unsigned": "teach_auth_unsigned", + "human_approver_unregistered": "teach_auth_unknown_signer", + "human_answer_signature_invalid": "teach_auth_bad_signature", + "attestation_method_mismatch": "teach_auth_method", +} + + +def verify( + target: Path, + *, + key_dir: Optional[Path], + root: Optional[str], + at_least: Any = None, +) -> Verification: + """Verify a teach authorization against the approver roster. + + ``root`` is the caller's pinned approver root and ``key_dir`` the key + directory holding the roster; nothing here reads the environment. The + signer must be on that roster now (so a removed key refuses), must be + the statement's ``authorizer`` with its ``authorizer_public_key``, and + must have signed exactly these statement bytes. + """ + + from openadapt_flow.approval_mode.human_attestation import ( + AttestationRefused, + HumanAttestation, + approvers, + refusal, + ) + + statement_path, sig_path = _paths(Path(target)) + try: + raw, data, statement = _load(statement_path) + except TeachAuthorizationError as exc: + return Verification(False, reason=exc.code, detail=str(exc)) + base = Verification( + False, + statement=statement, + sha256=hashlib.sha256(raw).hexdigest(), + approver_root=root, + ) + try: + sig_bytes = sig_path.read_bytes() + except FileNotFoundError: + return dataclasses.replace(base, reason="teach_auth_unsigned") + except OSError: + return dataclasses.replace(base, reason="teach_auth_malformed") + try: + attestation = HumanAttestation.model_validate_json(sig_bytes) + except ValueError: + return dataclasses.replace( + base, reason="teach_auth_malformed", detail="the signature file" + ) + base = dataclasses.replace( + base, + signature_sha256=hashlib.sha256(sig_bytes).hexdigest(), + method=attestation.method, + ) + if attestation.method not in SIGNING_METHODS: + return dataclasses.replace( + base, + reason="teach_auth_method", + detail=f"{attestation.method} can't sign a teach authorization", + ) + if attestation.signer != statement.authorizer: + return dataclasses.replace( + base, + reason="teach_auth_signer_mismatch", + detail="the signer isn't the statement's authorizer", + ) + code = refusal( + attestation, + domain=DOMAIN, + payload=data, + key_dir=Path(key_dir) if key_dir is not None else None, + root=root, + at_least=at_least, + ) + if code is not None: + return dataclasses.replace( + base, + reason=_ROSTER_REFUSALS.get(code, "teach_auth_identity_unverified"), + detail=code, + ) + try: + public = approvers(Path(key_dir), root, at_least=at_least).get( # type: ignore[arg-type] + attestation.signer + ) + except AttestationRefused as exc: + return dataclasses.replace( + base, reason="teach_auth_identity_unverified", detail=exc.code + ) + if public != statement.authorizer_public_key: + return dataclasses.replace( + base, + reason="teach_auth_signer_mismatch", + detail="the roster holds another key for the authorizer", + ) + return dataclasses.replace( + base, ok=True, signer=attestation.signer, signer_public_key=public + ) + + +def require( + target: Path, + *, + purpose: Purpose, + key_dir: Optional[Path], + root: Optional[str], + task_id: Optional[str] = None, + now: Optional[datetime] = None, + at_least: Any = None, +) -> Verification: + """A verified authorization fit for ``purpose`` now, or :class:`TeachAuthorizationError`.""" + + result = verify(target, key_dir=key_dir, root=root, at_least=at_least) + if not result.ok or result.statement is None: + raise TeachAuthorizationError(str(result.reason), result.detail) + st = result.statement + stamp = _utc_seconds(now) + if st.issued_at > stamp: + raise TeachAuthorizationError("teach_auth_not_yet_valid", st.issued_at) + if st.expires_at is not None and stamp >= st.expires_at: + raise TeachAuthorizationError("teach_auth_expired", st.expires_at) + _fits(result, purpose=purpose, task_id=task_id) + return result + + +def _fits( + result: Verification, *, purpose: Optional[str], task_id: Optional[str] +) -> None: + st = result.statement + assert st is not None + if purpose is not None and purpose not in st.purposes: + raise TeachAuthorizationError( + "teach_auth_purpose", f"it covers {', '.join(st.purposes)}, not {purpose}" + ) + if task_id is not None and task_id != st.task.task_id: + raise TeachAuthorizationError( + "teach_auth_task_mismatch", + f"authorized task {st.task.task_id}, asked for {task_id}", + ) + + +def pinned_trust( + key_dir: Optional[Path] = None, + approver_root: Optional[str] = None, + *, + env: Optional[Mapping[str, str]] = None, + config_only: bool = False, +) -> tuple[Path, Optional[str]]: + """The key directory and approver root a teach path checks against. + + ``key_dir`` defaults to ``$OPENADAPT_JIT_KEY_DIR`` (or + ``~/.openadapt/jit``). The root is ``approver_root``, else + ``OPENADAPT_APPROVER_ROOT`` (:func:`approver_roster.pinned_root`); two + that disagree refuse. ``None`` means unpinned, and verification refuses. + ``config_only`` (a process an agent runs, such as ``openadapt-agent + serve``): ``approver_root`` must come from the deployment config, and the + environment never counts. + """ + + from openadapt_flow.approval_mode import approver_roster as roster + from openadapt_flow.jit.lifecycle import default_key_dir + + try: + root = ( + roster.pinned_root(config_root=approver_root, config_only=True) + if config_only + else roster.pinned_root(approver_root, env=env) + ) + except roster.RosterError as exc: + raise TeachAuthorizationError( + "teach_auth_identity_unverified", exc.code + ) from exc + return (Path(key_dir) if key_dir is not None else default_key_dir()), root + + +# -- binding to a scope ----------------------------------------------------------- + + +def resolve_inputs( + statement: TeachAuthorizationV1, supplied: Optional[Mapping[str, Any]] +) -> dict[str, str]: + """The input values the run may use, checked against the statement. + + A spelled-out input needs no supplied value (a supplied one must equal + it). A digest-bound input needs a supplied value whose digest matches. + A supplied name the statement does not declare is refused. + """ + + supplied = {str(k): v for k, v in (supplied or {}).items() if v is not None} + unknown = sorted(set(supplied) - set(statement.inputs)) + if unknown: + raise TeachAuthorizationError( + "teach_auth_input_not_declared", ", ".join(unknown) + ) + out: dict[str, str] = {} + for name, bound in statement.inputs.items(): + if bound.value is not None: + if name in supplied and str(supplied[name]) != bound.value: + raise TeachAuthorizationError("teach_auth_input_mismatch", name) + out[name] = bound.value + continue + if name not in supplied: + raise TeachAuthorizationError("teach_auth_input_missing", name) + if value_sha256(str(supplied[name])) != bound.sha256: + raise TeachAuthorizationError("teach_auth_input_mismatch", name) + out[name] = str(supplied[name]) + return out + + +def _resolve( + bound: Bound, inputs: Mapping[str, str], supplied: Mapping[str, Any], name: str +) -> str: + if bound.value is not None: + return bound.value + if bound.input is not None: + return inputs[bound.input] + value = supplied.get(name) + if value is None or value_sha256(str(value)) != bound.sha256: + raise TeachAuthorizationError("teach_auth_input_mismatch", name) + return str(value) + + +def bind( + verification: Verification, + params: Optional[Mapping[str, Any]] = None, + *, + record_values: Optional[Mapping[str, Any]] = None, + write_adapter: Any = None, +) -> Any: + """The :class:`~openadapt_flow.teach_scope.TeachScope` a verified authorization allows. + + ``params`` are the run's input values (needed for digest-bound inputs); + ``record_values`` supply digest-bound identity, field and constant values + by name. ``write_adapter`` overrides loading the statement's adapter + reference (tests, an adapter object already built for the same app). + """ + + from openadapt_flow.teach_scope import TeachScope, load_write_adapter + + if not verification.ok or verification.statement is None: + raise TeachAuthorizationError(verification.reason or "teach_auth_malformed") + st = verification.statement + inputs = resolve_inputs(st, params) + extra = dict(record_values or {}) + identity = { + name: _resolve(b, inputs, extra, name) for name, b in st.writes.identity.items() + } + fields = { + name: _resolve(b, inputs, extra, name) for name, b in st.writes.fields.items() + } + constants = [_resolve(b, inputs, extra, "constant") for b in st.constants] + used = set(identity.values()) | set(fields.values()) + adapter = write_adapter + if adapter is None and st.writes.write_adapter: + try: + adapter = load_write_adapter(st.writes.write_adapter) + except Exception as exc: # import or construction errors + raise TeachAuthorizationError( + "teach_auth_adapter_unloadable", st.writes.write_adapter + ) from exc + return TeachScope( + task_id=st.task.task_id, + origins=tuple(st.origins), + record=identity, + fields=fields, + constants=tuple( + dict.fromkeys([*(v for v in inputs.values() if v not in used), *constants]) + ), + expected_rows=st.writes.expected_rows, + write_adapter=adapter, + allowed_write_paths=tuple(st.writes.allowed_write_paths), + forbidden_actions=frozenset(st.forbidden.actions), + forbidden_controls=tuple(st.forbidden.controls), + authorization=record(verification), + ) + + +def check_within(scope: Any, authorized: Any) -> None: + """Refuse a scope that allows anything the authorized scope does not. + + For paths where a scope is also assembled from flags or a task file: it + may narrow the authorization (fewer origins, fewer typeable strings), but + the record and field values must be the authorized ones. + """ + + from openadapt_flow.teach_scope import _same + + problems = [] + if not set(scope.origins) <= set(authorized.origins): + problems.append("origins") + if set(scope.record) != set(authorized.record) or any( + not _same(scope.record[k], authorized.record[k]) for k in scope.record + ): + problems.append("record") + if set(scope.fields) != set(authorized.fields) or any( + not _same(scope.fields[k], authorized.fields[k]) for k in scope.fields + ): + problems.append("fields") + if not set(scope.typeable) <= set(authorized.typeable): + problems.append("typeable text") + if scope.expected_rows != authorized.expected_rows: + problems.append("expected_rows") + if not set(scope.allowed_write_paths) <= set(authorized.allowed_write_paths): + problems.append("write paths") + if problems: + raise TeachAuthorizationError( + "teach_auth_scope_widened", + "the scope goes beyond the signed authorization: " + ", ".join(problems), + ) + + +def record(verification: Verification) -> dict[str, Any]: + """What a draft, candidate and bundle record about the authorization. No values.""" + + st = verification.statement + if st is None or verification.sha256 is None or not verification.ok: + raise TeachAuthorizationError("teach_auth_malformed") + return { + "schema": SCHEMA, + "sha256": verification.sha256, + "signature_sha256": verification.signature_sha256, + "task_id": st.task.task_id, + "authorizer": st.authorizer, + "authorizer_public_key": st.authorizer_public_key, + "method": verification.method, + "approver_root": verification.approver_root, + "record_kind": st.writes.record_kind, + "purposes": list(st.purposes), + "expires_at": st.expires_at, + } + + +def check_admission( + recorded: Optional[Mapping[str, Any]], + target: Optional[Path], + *, + key_dir: Optional[Path], + root: Optional[str], + purpose: Optional[Purpose] = None, + task_id: Optional[str] = None, + not_signed_by: Optional[str] = None, + at_least: Any = None, +) -> Verification: + """Verify, for admission, the authorization a draft names. + + ``recorded`` is the draft's ``teach_authorization`` entry. ``target`` is + the statement the draft kept (:func:`copy_into`). The statement's digest + must equal the recorded one, and its signature must verify against the + approver roster as it is now: a key removed since teaching refuses. + ``purpose`` and ``task_id``, when given, must be covered. + ``not_signed_by`` names a principal that may not be the signer (the + agent that recorded the draft). Expiry is not checked again: it bounds + when teaching may happen. + """ + + if not recorded or not isinstance(recorded, Mapping): + raise TeachAuthorizationError( + "teach_auth_not_recorded", + "the draft names no teach authorization; an agent-taught draft needs one", + ) + if target is None or not Path(target).is_file(): + raise TeachAuthorizationError("teach_auth_missing", "no statement kept with it") + result = verify(target, key_dir=key_dir, root=root, at_least=at_least) + if not result.ok: + raise TeachAuthorizationError(str(result.reason), result.detail) + if result.sha256 != recorded.get("sha256"): + raise TeachAuthorizationError( + "teach_auth_digest_mismatch", + "the statement kept with it is not the one it recorded", + ) + if not_signed_by and result.signer == not_signed_by: + raise TeachAuthorizationError( + "teach_auth_agent_signer", + "the recording agent signed its own authorization", + ) + _fits(result, purpose=purpose, task_id=task_id) + return result + + +def copy_into(source: Path, folder: Path) -> Path: + """Copy the statement and its signature into ``folder``; return the statement path. + + Refuses to replace a different statement already there. + """ + + statement, sig = _paths(Path(source)) + folder = Path(folder) + out = folder / STATEMENT_FILE + data, signature = statement.read_bytes(), sig.read_bytes() + if out.exists(): + if ( + out.read_bytes() != data + or (folder / SIGNATURE_FILE).read_bytes() != signature + ): + raise TeachAuthorizationError( + "teach_auth_digest_mismatch", f"{folder} holds another authorization" + ) + return out + folder.mkdir(parents=True, exist_ok=True) + out.write_bytes(data) + (folder / SIGNATURE_FILE).write_bytes(signature) + return out + + +def render(verification: Verification) -> str: + """The plain-text summary a person reads before signing or admitting.""" + + from openadapt_flow.authorization_summary import clean + + st = verification.statement + if st is None: + return "no statement" + + def show(name: str, b: Bound) -> str: + if b.value is not None: + return f"{clean(name)} = {clean(b.value)!r}" + if b.input is not None: + return f"{clean(name)} = input {b.input}" + return f"{clean(name)} = the value with sha256 {b.sha256[:12]}… (supplied at teach time)" # type: ignore[index] + + w = st.writes + lines = [ + "You are authorizing an agent to teach (or repair) this task", + f" Authorizer {clean(st.authorizer)} (approver key {st.authorizer_public_key[:16]}…)", + f" Task {clean(st.task.task_id)}: {clean(st.task.summary)}", + f" Covers {', '.join(st.purposes)}", + f" Valid from {st.issued_at}" + + (f" until {st.expires_at}" if st.expires_at else " (no expiry)"), + "", + "The agent may", + f" reach only {', '.join(st.origins)}", + *(f" type input {show(n, b)}" for n, b in st.inputs.items()), + *(f" type {show('constant', b)}" for b in st.constants), + "", + f"Exactly {w.expected_rows} {clean(w.record_kind)} write" + + ("s" if w.expected_rows != 1 else ""), + *(f" identity {show(n, b)}" for n, b in w.identity.items()), + *(f" field {show(n, b)}" for n, b in w.fields.items()), + f" checked by {clean(w.write_adapter or 'the record read-back only')}", + "", + "Never", + " any other origin, record, field, value or typed text; pasting", + *(f" {a.replace('_', ' ')}" for a in st.forbidden.actions), + *(f" click {clean(c)!r}" for c in st.forbidden.controls), + "", + f"Statement sha256 {verification.sha256}", + ] + return "\n".join(lines) + + +@dataclass(frozen=True) +class TeachAuthority: + """Where an authoring path gets its scope: a signed statement and the roster. + + Args: + statement: The ``teach-authorization.json`` (or its folder); its + ``.sig`` sits beside it. + key_dir: The key directory holding the approver roster. Defaults to + ``$OPENADAPT_JIT_KEY_DIR`` (or ``~/.openadapt/jit``). + approver_root: The approver root to pin. Defaults to + ``OPENADAPT_APPROVER_ROOT``; without one, verification refuses. + write_adapter: An adapter object to use instead of loading the + statement's ``write_adapter`` reference. + config_only: ``approver_root`` came from the deployment config and + the environment never counts (a process an agent runs). + at_least: Roster heads recorded outside the key directory (the + deployment config's ``approvers.head``); the roster must extend + them. + """ + + statement: Path + key_dir: Optional[Path] = None + approver_root: Optional[str] = None + write_adapter: Any = None + config_only: bool = False + at_least: Any = None + + def kept_in(self, folder: Path) -> "TeachAuthority": + """Copy the statement into ``folder`` and return an authority over the copy. + + The copy is what gets verified, bound and kept beside the draft, so + a file swapped after the check can't be the one the draft names. + """ + + return dataclasses.replace(self, statement=copy_into(self.statement, folder)) + + def trust(self) -> tuple[Path, Optional[str]]: + return pinned_trust( + self.key_dir, self.approver_root, config_only=self.config_only + ) + + def verify( + self, *, purpose: Purpose, task_id: Optional[str] = None + ) -> Verification: + key_dir, root = self.trust() + return require( + self.statement, + purpose=purpose, + key_dir=key_dir, + root=root, + task_id=task_id, + at_least=self.at_least, + ) + + def scope( + self, + *, + purpose: Purpose, + task_id: Optional[str] = None, + params: Optional[Mapping[str, Any]] = None, + record_values: Optional[Mapping[str, Any]] = None, + ) -> Any: + """The bound :class:`~openadapt_flow.teach_scope.TeachScope`, or an error.""" + + return bind( + self.verify(purpose=purpose, task_id=task_id), + params, + record_values=record_values, + write_adapter=self.write_adapter, + ) + + +def narrowed(scope: Any, authorized: Any) -> Any: + """``scope`` (from flags or a task file) under ``authorized``'s signature. + + Refuses a scope wider than the authorization (:func:`check_within`); the + result keeps the narrower scope's limits, adds the authorization's + forbidden list and records the authorization. + """ + + check_within(scope, authorized) + return dataclasses.replace( + scope, + # The adapter the person signed for wins; a caller's may only add one. + write_adapter=authorized.write_adapter + if authorized.write_adapter is not None + else scope.write_adapter, + forbidden_actions=frozenset(scope.forbidden_actions) + | frozenset(authorized.forbidden_actions), + forbidden_controls=tuple( + dict.fromkeys([*scope.forbidden_controls, *authorized.forbidden_controls]) + ), + authorization=authorized.authorization, + ) diff --git a/packages/flow/openadapt_flow/teach_authorization_fixture.py b/packages/flow/openadapt_flow/teach_authorization_fixture.py new file mode 100644 index 0000000000..6ef5c2d289 --- /dev/null +++ b/packages/flow/openadapt_flow/teach_authorization_fixture.py @@ -0,0 +1,202 @@ +"""A throwaway approver for tests, demos and evaluations. + +A real teach authorization is signed by a person with their approver key +(:mod:`openadapt_flow.teach_authorization`). Tests, the MockMed demos and the +injection evaluation need one without a person at the keyboard, so +:class:`LocalAuthorizer` makes an Ed25519 approver key in memory, starts (or +joins) a real signed approver roster in a key directory, and signs statements +with the real approver signer. Its root is what a run pins: anything it signs +is trusted only where that root is pinned. Never pin it in a deployment. +""" + +from __future__ import annotations + +import secrets +from datetime import datetime, timedelta, timezone +from pathlib import Path +from typing import Any, Mapping, Optional + +from openadapt_flow import teach_authorization as ta + +__all__ = ["LocalAuthorizer", "demo_authority"] + + +class LocalAuthorizer: + """An in-memory approver key on a real approver roster in ``key_dir``. + + The first one in a key directory starts the roster (it is the root); + pass ``added_by`` to add another approver with that one's signature. + ``register=False`` makes a key that is on no roster. + """ + + def __init__( + self, + key_dir: Path, + principal: str = "teach-authorizer", + *, + added_by: "Optional[LocalAuthorizer]" = None, + method: str = "person_key", + register: bool = True, + ) -> None: + from cryptography.hazmat.primitives.asymmetric.ed25519 import ( + Ed25519PrivateKey, + ) + + from openadapt_flow.approval_mode import approver_roster as roster + from openadapt_flow.jit.authority import public_hex + + self.key_dir = Path(key_dir) + self.principal = principal + self.method = method + self._key = Ed25519PrivateKey.generate() + self.public_key = public_hex(self._key) + self.root: str = added_by.root if added_by is not None else self.public_key + if not register: + return + self.key_dir.mkdir(parents=True, exist_ok=True) + if added_by is None: + roster.init_root( + self.key_dir, + principal=principal, + public_key=self.public_key, + sign=roster.file_key_signer(self._key), + ) + else: + roster.add_approver( + self.key_dir, + self.root, + principal=principal, + public_key=self.public_key, + signed_by=added_by.principal, + sign=roster.file_key_signer(added_by._key), + ) + + def remove(self, other: "LocalAuthorizer") -> None: + """Remove (revoke) ``other`` from the roster with this approver's signature.""" + + from openadapt_flow.approval_mode import approver_roster as roster + + roster.remove_approver( + self.key_dir, + self.root, + principal=other.principal, + public_key=other.public_key, + signed_by=self.principal, + sign=roster.file_key_signer(self._key), + ) + + def signer(self) -> Any: + from openadapt_flow.approval_mode.human_attestation import ApproverSigner + + return ApproverSigner( + principal_id=self.principal, + method=self.method, # type: ignore[arg-type] + signing=self._key, + ) + + def authority(self, statement: Path, **kw: Any) -> ta.TeachAuthority: + """A :class:`TeachAuthority` over ``statement`` that pins this roster.""" + + return ta.TeachAuthority( + Path(statement), key_dir=self.key_dir, approver_root=self.root, **kw + ) + + def issue( + self, + folder: Path, + spec: Mapping[str, Any], + *, + now: Optional[datetime] = None, + valid_for: timedelta = timedelta(days=1), + authorizer: Optional[str] = None, + ) -> Path: + """Write and sign a statement for ``spec`` in ``folder``; return its path.""" + + now = now or datetime.now(timezone.utc) + statement = ta.TeachAuthorizationV1.model_validate( + { + **dict(spec), + "schema": ta.SCHEMA, + "authorizer": authorizer or self.principal, + "authorizer_public_key": self.public_key, + "issued_at": ta._utc_seconds(now - timedelta(seconds=5)), + "expires_at": ta._utc_seconds(now + valid_for), + "nonce": secrets.token_hex(32), + } + ) + folder = Path(folder) + folder.mkdir(parents=True, exist_ok=True) + path = folder / ta.STATEMENT_FILE + path.write_text(ta.dumps(statement), encoding="utf-8") + self.sign_file(path, as_principal=authorizer) + return path + + def sign_file( + self, statement_path: Path, *, as_principal: Optional[str] = None + ) -> Path: + """Sign with the real approver signer, as ``teach-authorization sign`` does. + + Writes the attestation directly (not through :func:`ta.sign`, which + checks the signer first) so a test can sign as a principal the + roster doesn't hold for this key. + """ + + from openadapt_flow.approval_mode.human_attestation import ApproverSigner + from openadapt_flow.offline_seal import loads_strict + + data = loads_strict(Path(statement_path).read_bytes()) + signer = ApproverSigner( + principal_id=as_principal or self.principal, + method=self.method, # type: ignore[arg-type] + signing=self._key, + ) + attestation = signer.attest(ta.DOMAIN, data) + sig = Path(statement_path).with_name(Path(statement_path).name + ".sig") + sig.write_text(attestation.model_dump_json() + "\n", encoding="utf-8") + return sig + + @staticmethod + def digest(value: str) -> str: + return ta.value_sha256(value) + + def __repr__(self) -> str: # never show key material + return f"LocalAuthorizer({self.principal}, {self.public_key[:12]})" + + +def demo_authority( + work_dir: Path, + *, + task_id: str, + url: str, + record: Mapping[str, Any], + inputs: Mapping[str, Any], + fields: Optional[Mapping[str, Any]] = None, + constants: Any = (), + record_kind: str = "record", +) -> ta.TeachAuthority: + """A teach authorization a throwaway approver signs, for demos and benchmarks. + + Starts a new approver roster in a fresh folder under ``work_dir``, signs + a statement for exactly this run (the task, the app's origin, the record, + the inputs' values and any other typeable text), and returns a + :class:`TeachAuthority` that pins that roster. Admission checks it like + any other. Never use it where a person must authorize the teaching. + """ + + from openadapt_flow.teach_scope import origin_of + + folder = Path(work_dir) / f"demo-teach-authorization-{secrets.token_hex(6)}" + person = LocalAuthorizer(folder / "approvers", "demo-authorizer") + spec = { + "task": {"task_id": task_id, "summary": f"Demo run of {task_id}"}, + "purposes": ["teach", "repair"], + "origins": [str(origin_of(url))], + "inputs": {str(k): {"value": str(v)} for k, v in inputs.items()}, + "constants": [{"value": str(c)} for c in dict.fromkeys(map(str, constants))], + "writes": { + "record_kind": record_kind, + "identity": {str(k): {"value": str(v)} for k, v in record.items()}, + "fields": {str(k): {"value": str(v)} for k, v in (fields or {}).items()}, + }, + } + return person.authority(person.issue(folder, spec)) diff --git a/packages/flow/openadapt_flow/teach_scope.py b/packages/flow/openadapt_flow/teach_scope.py index ebfb8574e4..4dbadb45ee 100644 --- a/packages/flow/openadapt_flow/teach_scope.py +++ b/packages/flow/openadapt_flow/teach_scope.py @@ -208,6 +208,14 @@ class TeachScope: allowed_write_paths: URL paths on a declared origin that may take a write the adapter does not attribute (a sign-in form). Exact match; no prefixes. + forbidden_actions: Actions the agent may never take, from + :data:`~openadapt_flow.teach_authorization.FORBIDDEN_ACTIONS` + (paste is always refused). + forbidden_controls: Labels of controls the agent may never click + (compared after whitespace and case normalization). + authorization: The signed teach authorization this scope was bound + from (:func:`openadapt_flow.teach_authorization.record`), or + ``None`` for a scope nobody signed. """ task_id: str @@ -218,6 +226,9 @@ class TeachScope: expected_rows: int = 1 write_adapter: Optional[WriteAdapter] = field(default=None, compare=False) allowed_write_paths: tuple[str, ...] = () + forbidden_actions: frozenset[str] = frozenset() + forbidden_controls: tuple[str, ...] = () + authorization: Optional[Mapping[str, Any]] = field(default=None, compare=False) def __post_init__(self) -> None: if not self.task_id: @@ -246,6 +257,27 @@ def __post_init__(self) -> None: for path in self.allowed_write_paths: if not path.startswith("/"): raise ValueError(f"allowed write path must start with '/': {path!r}") + from openadapt_flow.teach_authorization import FORBIDDEN_ACTIONS + + unknown = set(self.forbidden_actions) - set(FORBIDDEN_ACTIONS) + if unknown: + raise ValueError("unknown forbidden actions: " + ", ".join(sorted(unknown))) + object.__setattr__(self, "forbidden_actions", frozenset(self.forbidden_actions)) + object.__setattr__( + self, + "forbidden_controls", + tuple(normalize_structured(c) for c in self.forbidden_controls if c), + ) + if self.authorization is not None: + object.__setattr__( + self, "authorization", MappingProxyType(dict(self.authorization)) + ) + + @property + def authorized(self) -> bool: + """True when a person signed this scope (it was bound from an authorization).""" + + return self.authorization is not None @classmethod def from_guard_task( @@ -326,6 +358,8 @@ def request_refusal( return None if not self.allows_origin(url): return "egress_origin_not_declared" + if method == "DELETE" and "delete_request" in self.forbidden_actions: + return "forbidden_action" if method not in WRITE_METHODS or self.write_adapter is None: return None try: @@ -394,6 +428,11 @@ def to_dict(self) -> dict[str, Any]: if self.write_adapter is not None else None, "allowed_write_paths": list(self.allowed_write_paths), + "forbidden_actions": sorted(self.forbidden_actions), + "forbidden_controls": len(self.forbidden_controls), + "authorization": dict(self.authorization) + if self.authorization is not None + else None, } def describe(self) -> str: @@ -413,6 +452,7 @@ class ScopeViolation(UnsupportedAction): """An action or request went outside the declared scope. ``code`` is closed. Codes: ``typed_value_not_declared``, ``paste_refused``, + ``forbidden_action``, ``control_forbidden``, ``link_origin_not_declared``, ``egress_origin_not_declared``, ``navigated_off_scope``, ``write_wrong_record``, ``write_field_not_declared``, ``write_value_not_declared``, @@ -469,6 +509,31 @@ def to_dict(self) -> dict[str, Any]: }""" +_CONTROL_LABEL = """([x, y]) => { + const el = document.elementFromPoint(x, y); + if (!el) return null; + const c = el.closest('button, a, input, select, [role], label, summary') || el; + const text = c.getAttribute('aria-label') || c.getAttribute('title') || + (c.tagName === 'INPUT' ? (c.value || c.getAttribute('placeholder') || '') : '') || + c.innerText || c.textContent || ''; + return text.trim().slice(0, 300); +}""" + + +def control_label_at(page: Any, x: int, y: int) -> Optional[str]: + """The accessible label of the control under ``(x, y)``. + + ``""`` when there is no control or it has no label; ``None`` when the + probe failed (the page broke or overrode it), which the guard refuses. + """ + + try: + label = page.evaluate(_CONTROL_LABEL, [int(x), int(y)]) + except Exception: # noqa: BLE001 - the probe failed: the label is unknown + return None + return str(label) if label is not None else "" + + def link_target_at(page: Any, x: int, y: int) -> Optional[dict[str, str]]: """The link or form submit under ``(x, y)``: ``{"kind", "url"}``, or ``None``. @@ -512,6 +577,24 @@ def install(self, page: Any) -> None: self._page = page page.context.route("**/*", self._route) + forbidden = self.scope.forbidden_actions + if "new_window" in forbidden: + page.context.on( + "page", + lambda _p: self._violate( + "forbidden_action", "the page opened a new window or tab" + ), + ) + if "download" in forbidden: + page.on( + "download", + lambda _d: self._violate("forbidden_action", "a download started"), + ) + if "file_upload" in forbidden: + page.on( + "filechooser", + lambda _f: self._violate("forbidden_action", "a file chooser opened"), + ) def _route(self, route: Any) -> None: request = route.request @@ -562,7 +645,35 @@ def check_ops( """Refuse an action before any of it runs. Raises :class:`ScopeViolation`.""" typeable = self.scope.typeable + forbidden = self.scope.forbidden_actions for op in ops: + if op.kind in ("drag", "right_click") and op.kind in forbidden: + self._raise( + "forbidden_action", + f"the authorization forbids {op.kind.replace('_', ' ')}", + ) + if ( + op.kind in ("click", "double_click", "right_click") + and page is not None + and self.scope.forbidden_controls + ): + at = (op.x, op.y) if op.x is not None and op.y is not None else pointer + label = control_label_at(page, at[0], at[1]) if at else "" + if label is None: + # Fail closed: a page that breaks the probe could hide a + # forbidden control behind it. + self._raise( + "control_forbidden", + "the clicked control's label could not be read, and the " + "authorization forbids some controls", + ) + if label and ( + normalize_structured(label) in self.scope.forbidden_controls + ): + self._raise( + "control_forbidden", + "the clicked control is one the authorization forbids", + ) if op.kind == "type": if (op.text or "") not in typeable: self._raise( diff --git a/packages/flow/tests/_teach_authorization_support.py b/packages/flow/tests/_teach_authorization_support.py new file mode 100644 index 0000000000..d5288002a2 --- /dev/null +++ b/packages/flow/tests/_teach_authorization_support.py @@ -0,0 +1,203 @@ +"""Give a test's agent-taught draft a real, person-signed teach authorization. + +Admission refuses a JIT draft without one +(:func:`openadapt_flow.jit.lifecycle.teach_authorization_gate`). Tests here +use the real pieces: a real approver roster in the test's key directory, a +real approver signature (:class:`LocalAuthorizer`), the real verifier, and +the root pinned in ``OPENADAPT_APPROVER_ROOT`` as an operator would. The +roster's root is shared with ``tests/_approver_keys.py``, so tests that also +answer saves as a person add their approvers to the same roster. +""" + +from __future__ import annotations + +from pathlib import Path +from typing import Any, Mapping, Optional, Sequence + +from openadapt_flow import teach_authorization as ta +from openadapt_flow.approval_mode.approver_roster import ROOT_ENV +from openadapt_flow.teach_authorization_fixture import LocalAuthorizer + +AUTHORIZER = "teach-lead" + + +def authorizer(key_dir: Path, monkeypatch: Any = None) -> LocalAuthorizer: + """The teach authorizer (an approver the root added) in ``key_dir``; pins the root.""" + + key_dir = Path(key_dir) + found = _AUTHORIZERS.get(key_dir) + if found is None: + found = LocalAuthorizer(key_dir, AUTHORIZER, added_by=roster_root(key_dir)) + _AUTHORIZERS[key_dir] = found + patch = monkeypatch or (_MONKEYPATCH[0] if _MONKEYPATCH else None) + if patch is not None: + patch.setenv(ROOT_ENV, found.root) + return found + + +_AUTHORIZERS: dict[Path, LocalAuthorizer] = {} + + +def roster_root(key_dir: Path) -> LocalAuthorizer: + """The roster's root approver in ``key_dir`` (shared with _approver_keys).""" + + from tests import _approver_keys + + key_dir = Path(key_dir) + root = _approver_keys._ROOTS.get(key_dir) + if root is None: + made = LocalAuthorizer(key_dir, "roster-admin") + _approver_keys._ROOTS[key_dir] = (made.principal, made._key, made.public_key) + return made + # A roster the approver fixture started: act as its root. + joined = LocalAuthorizer.__new__(LocalAuthorizer) + joined.key_dir, joined.principal = key_dir, root[0] + joined._key, joined.public_key, joined.root = root[1], root[2], root[2] + joined.method = "person_key" + return joined + + +#: The running test's monkeypatch (tests/conftest.py sets it), so the root is +#: pinned for that test only. +_MONKEYPATCH: list[Any] = [] + + +def spec( + task_id: str, + *, + origins: Sequence[str] = ("http://127.0.0.1:9",), + record: Optional[Mapping[str, str]] = None, + fields: Optional[Mapping[str, str]] = None, + inputs: Optional[Mapping[str, str]] = None, + constants: Sequence[str] = (), + record_kind: str = "note", + purposes: Sequence[str] = ("teach", "repair"), +) -> dict[str, Any]: + """A statement spec with spelled-out values.""" + + return { + "task": {"task_id": task_id, "summary": f"Synthetic task {task_id}"}, + "purposes": list(purposes), + "origins": list(origins), + "inputs": {k: {"value": v} for k, v in (inputs or {}).items()}, + "constants": [{"value": c} for c in constants], + "writes": { + "record_kind": record_kind, + "identity": { + k: {"value": v} for k, v in (record or {"type": "note"}).items() + }, + "fields": {k: {"value": v} for k, v in (fields or {}).items()}, + }, + } + + +def issue( + key_dir: Path, folder: Path, task_id: str, monkeypatch: Any = None, **kw: Any +) -> Path: + """A signed statement for ``task_id`` in ``folder``; return its path.""" + + return authorizer(key_dir, monkeypatch).issue(folder, spec(task_id, **kw)) + + +def authority( + key_dir: Path, folder: Path, task_id: str, monkeypatch: Any = None, **kw: Any +) -> ta.TeachAuthority: + """A :class:`TeachAuthority` for compile_agent, the pipeline or MCP teach.""" + + existing = Path(folder) / ta.STATEMENT_FILE + statement = ( + existing + if existing.is_file() + else issue(key_dir, folder, task_id, monkeypatch, **kw) + ) + return authorizer(key_dir, monkeypatch).authority(statement) + + +def mockmed_note( + key_dir: Path, + folder: Path, + task_id: str, + base_url: str, + *, + note: str, + record: Mapping[str, str], + constants: Sequence[str] = (), + monkeypatch: Any = None, +) -> ta.TeachAuthority: + """The authorization a person signs for "add this note to this MockMed patient".""" + + from openadapt_flow.teach_scope import origin_of + + return authority( + key_dir, + folder, + task_id, + monkeypatch, + origins=[str(origin_of(base_url))], + record=record, + fields={"note": note}, + inputs={"note": note}, + constants=constants, + record_kind="encounter", + ) + + +def draft_fields( + program: Path, key_dir: Path, task_id: str, monkeypatch: Any = None, **kw: Any +) -> dict[str, Any]: + """Candidate fields for a hand-built draft, as a scoped capture writes them. + + Signs a statement, keeps it beside the draft, and returns the + ``teach_authorization`` entry and a ``teach_scope`` block whose scope + names it and whose read-back verified. Add them to the candidate before + :func:`~openadapt_flow.jit.lifecycle.sign_recorder`. + """ + + from openadapt_flow.jit.lifecycle import TEACH_AUTHORIZATION_DIR + + statement = issue(key_dir, Path(program) / "_issued", task_id, monkeypatch, **kw) + kept = ta.copy_into(statement, Path(program) / TEACH_AUTHORIZATION_DIR) + found = authorizer(key_dir) + record = ta.record( + ta.require(kept, purpose="teach", key_dir=key_dir, root=found.root) + ) + return { + "teach_authorization": record, + "teach_scope": { + "scope": {"task_id": task_id, "authorization": record}, + "readback": {"status": "verified"}, + }, + } + + +def repair_authorization( + store_root: Path, + key_dir: Path, + task_id: str = "dialog-fixture-save", + monkeypatch: Any = None, + *, + purposes: Sequence[str] = ("repair",), +) -> dict[str, Any]: + """A signed authorization to repair ``task_id``, kept in the repair store. + + Returns the record a calling agent's repair provenance carries. Pins the + key directory (``OPENADAPT_JIT_KEY_DIR``) and the root, as an operator's + approval host would. + """ + + from openadapt_flow.jit.lifecycle import KEY_DIR_ENV + from openadapt_flow.repair import RepairStore + + statement = issue( + key_dir, + Path(store_root).parent / "_repair_authorization", + task_id, + monkeypatch, + purposes=purposes, + ) + patch = monkeypatch or (_MONKEYPATCH[0] if _MONKEYPATCH else None) + if patch is not None: + patch.setenv(KEY_DIR_ENV, str(key_dir)) + found = authorizer(key_dir) + verification = ta.verify(statement, key_dir=key_dir, root=found.root) + return RepairStore(store_root).keep_teach_authorization(verification, statement) diff --git a/packages/flow/tests/conftest.py b/packages/flow/tests/conftest.py index 145f115e44..71d2626feb 100644 --- a/packages/flow/tests/conftest.py +++ b/packages/flow/tests/conftest.py @@ -43,6 +43,18 @@ def pytest_sessionfinish(session: pytest.Session, exitstatus: int) -> None: check_and_report(session, exitstatus, REPO_ROOT) +@pytest.fixture(autouse=True) +def _teach_authorization_env(monkeypatch: pytest.MonkeyPatch): + """Lets tests/_teach_authorization_support.py pin an approver root for one test.""" + + from tests import _teach_authorization_support as support + + support._MONKEYPATCH[:] = [monkeypatch] + yield + support._MONKEYPATCH.clear() + support._AUTHORIZERS.clear() + + @pytest.fixture(autouse=True) def _isolate_durable_authority(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: """Keep each test's monotonic run authority independent and disposable.""" diff --git a/packages/flow/tests/e2e/test_agent_guard_e2e.py b/packages/flow/tests/e2e/test_agent_guard_e2e.py index cea88866eb..62075217b5 100644 --- a/packages/flow/tests/e2e/test_agent_guard_e2e.py +++ b/packages/flow/tests/e2e/test_agent_guard_e2e.py @@ -97,19 +97,18 @@ def test_guard_blocks_wrong_record_then_run_two_is_a_tool(tmp_path: Path) -> Non assert handoff["inputs"] == ["note"] assert handoff["expect"] == {"patient_id": "p1", "type": "Triage"} - second = bench.run2( - app, - Path(handoff["task_dir"]), - files["verify"], - teach_dir, - tmp_path / "taught-runs", - ) - assert second["admitted"], second - by_name = {r["scenario"]: r for r in second["runs"]} - assert by_name["ok"]["outcome"] == "clean" - assert by_name["ok"]["model_calls"] == 0 - for fault in ("no_persist", "partial", "duplicate", "wrong_record", "stale"): - assert by_name[fault]["outcome"] == "caught_by_readback", by_name[fault] + # The airbag handoff can't carry a person's teach authorization yet, + # so admission refuses its draft (docs/TEACH_AUTHORIZATION.md). + from openadapt_flow.jit import AdmissionError + + with pytest.raises(AdmissionError, match="teach_auth_not_recorded"): + bench.run2( + app, + Path(handoff["task_dir"]), + files["verify"], + teach_dir, + tmp_path / "taught-runs", + ) finally: app.stop() diff --git a/packages/flow/tests/e2e/test_browser_teach_write_e2e.py b/packages/flow/tests/e2e/test_browser_teach_write_e2e.py index b64404d384..bc99ea108d 100644 --- a/packages/flow/tests/e2e/test_browser_teach_write_e2e.py +++ b/packages/flow/tests/e2e/test_browser_teach_write_e2e.py @@ -1,57 +1,25 @@ -"""Teach a write once, then call it as a tool that returns a verified receipt. +"""Taught browser writes are refused. -Everything is real except the agent: the MCP SDK client talks to the real -server, which drives headless Chromium against the example app on -127.0.0.1. Run 1 signs in with values the server holds, fills a New Order -form and saves it. The write contract reads the app's order store -(``GET /api/orders``), so every result comes from the store, not the screen. -The agent is scripted: it picks refs from the listings by label. No model is -called. - -It asserts: - -* teach_finish verifies run 1's order in the store before it adds the tool; -* the taught tool, called with a new order, saves it and returns a verified - receipt; -* an app that answers "saved" but keeps nothing, or keeps the order twice, - gets a refusal, never done; -* the sign-in values never appear in a page listing, a result or the program. +Browser teach refuses taught writes until a person's signed teach +authorization is bound to them (docs/TEACH_AUTHORIZATION.md). Against the +real example app: ``teach_start`` with a write contract is refused before a +browser opens, and a write tool already on disk doesn't run. Nothing is +written to the app. """ from __future__ import annotations import json -import re import urllib.request from pathlib import Path -from typing import Any, Iterator, Optional +from typing import Any, Iterator -import anyio import pytest -pytest.importorskip("mcp") -pytest.importorskip("playwright") - -from mcp import ClientSession # noqa: E402 -from mcp.shared.memory import create_client_server_memory_streams # noqa: E402 - -from openadapt_flow.agent.browser_teach import BrowserTeachBridge # noqa: E402 -from openadapt_flow.agent.example_app import ( # noqa: E402 - DEMO_PASSWORD, - DEMO_USER, - serve, -) -from openadapt_flow.agent.mcp import build_server # noqa: E402 -from openadapt_flow.verify import WriteContract # noqa: E402 - -FIRST = {"customer": "Acme Ltd", "product": "Desk lamp", "units": "3"} -SECOND = {"customer": "Birch Co", "product": "Office chair", "units": "5"} - - -@pytest.fixture(autouse=True) -def _seal_home(tmp_path, monkeypatch): - # Receipts are sealed; keep the test's Seal key out of the home folder. - monkeypatch.setenv("OPENADAPT_SEAL_HOME", str(tmp_path / "seal")) +from openadapt_flow.agent import browser_teach as bt +from openadapt_flow.agent.bridge import BridgeError +from openadapt_flow.agent.example_app import serve +from openadapt_flow.verify import WriteContract @pytest.fixture @@ -61,20 +29,15 @@ def app() -> Iterator[str]: stop() -def _get(url: str) -> Any: - with urllib.request.urlopen(url, timeout=5) as response: - body = response.read() - try: - return json.loads(body) - except ValueError: - return body.decode() +def _orders(base: str) -> Any: + with urllib.request.urlopen(f"{base}/api/orders", timeout=5) as response: + return json.loads(response.read())["records"] def _contract(base: str) -> WriteContract: return WriteContract.from_dict( { "schema_version": "openadapt.verify/v1", - # The requests a taught order may send: the sign-in and the save. "writes": ["POST /login", "POST /orders"], "verifier": { "kind": "rest", @@ -85,11 +48,7 @@ def _contract(base: str) -> WriteContract: "effects": [ { "kind": "record_written", - "match": { - "customer": {"param": "customer"}, - "product": {"param": "product"}, - "units": {"param": "units"}, - }, + "match": {"customer": {"param": "customer"}}, "expected_count": 1, "count_new_only": True, "timeout_s": 1.0, @@ -100,174 +59,41 @@ def _contract(base: str) -> WriteContract: ) -def _ref(listing: str, label: str) -> str: - for line in listing.splitlines(): - found = re.match(r"\[(e\d+)\] \S+ \"" + re.escape(label) + '"', line) - if found: - return found.group(1) - raise AssertionError(f"no {label!r} in listing:\n{listing}") - - -async def _session(server: Any, steps: Any) -> Any: - async with create_client_server_memory_streams() as (client, transport): - async with anyio.create_task_group() as tasks: - tasks.start_soon( - server.run, *transport, server.create_initialization_options() - ) - async with ClientSession(*client) as session: - await session.initialize() - - async def call(name: str, args: Optional[dict] = None) -> dict: - result = await session.call_tool(name, args or {}) - text = result.content[0].text - payload = json.loads(text) - if getattr(result, "isError", None) or getattr( - result, "is_error", None - ): - return {"_error": payload.get("error"), "_text": text} - payload["_text"] = text - return payload - - out = await steps(call) - tasks.cancel_scope.cancel() - return out - - -def test_teach_a_write_then_call_it_verified( - app: str, tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: - monkeypatch.setenv("PORTAL_USER", DEMO_USER) - monkeypatch.setenv("PORTAL_PASSWORD", DEMO_PASSWORD) - bridge = BrowserTeachBridge( - tmp_path / "tools", - tmp_path / "files", - contracts={"order": _contract(app)}, - secrets={"username": "PORTAL_USER", "password": "PORTAL_PASSWORD"}, - verify_dir=tmp_path / "verify", +def test_a_taught_write_is_refused_and_writes_nothing(app: str, tmp_path: Path) -> None: + before = _orders(app) + bridge = bt.BrowserTeachBridge( + tmp_path / "tools", tmp_path / "files", contracts={"order": _contract(app)} ) - server = build_server(teach=bridge) - seen: list[str] = [] - - async def steps(call): - out: dict[str, Any] = {} - started = await call( + with pytest.raises(BridgeError, match="taught writes are refused"): + bridge.dispatch( "teach_start", { "name": "add_order", "url": f"{app}/login", - "task": "Add an order for a customer", + "task": "Add an order", "contract": "order", - "inputs": FIRST, - }, - ) - assert started.get("status") == "recording", started - assert started["sign_in_inputs"] == ["password", "username"] - page = started["page"] - seen.append(started["_text"]) - - async def act(**args: Any) -> None: - nonlocal page - result = await call("teach_act", args) - assert "_error" not in result, (args, result) - seen.append(result["_text"]) - page = result["page"] - - await act(action="type", ref=_ref(page, "Username"), input="username") - await act(action="type", ref=_ref(page, "Password"), input="password") - await act(action="click", ref=_ref(page, "Sign in")) - await act(action="type", ref=_ref(page, "Customer"), input="customer") - await act(action="select", ref=_ref(page, "Product"), input="product") - await act(action="type", ref=_ref(page, "Units"), input="units") - await act(action="click", ref=_ref(page, "Save order")) - out["finished"] = await call("teach_finish", {}) - seen.append(out["finished"]["_text"]) - - out["second"] = await call("taught_add_order", {"inputs": SECOND}) - _get(f"{app}/_fault?mode=no_persist") - out["dropped"] = await call( - "taught_add_order", - { - "inputs": { - "customer": "Cedar Inc", - "product": "USB-C dock", - "units": "2", - } + "inputs": {"customer": "Acme Ltd"}, }, ) - _get(f"{app}/_fault?mode=twice") - out["doubled"] = await call( - "taught_add_order", - { - "inputs": { - "customer": "Dune LLC", - "product": "Monitor arm", - "units": "4", - } - }, - ) - _get(f"{app}/_fault?mode=") - for key in ("second", "dropped", "doubled"): - seen.append(out[key]["_text"]) - return out - - out = anyio.run(_session, server, steps) - - finished = out["finished"] - assert finished["status"] == "ready", finished - assert finished["new_tool"] == "taught_add_order" - assert finished["run_1"]["verified"] is True - assert finished["writes_sent"] == ["POST /login", "POST /orders"] - - second = out["second"] - assert second["outcome"] == "done", second - assert second["verified"] is True and len(second["receipt_sha256"]) == 64 - assert second["model_calls"] == 0 - - dropped = out["dropped"] - assert dropped["outcome"] == "needs_review", dropped - assert dropped["verified"] is False and dropped["retry_safe"] is False - assert dropped["verdict"] == "RECONCILIATION_REQUIRED" - - doubled = out["doubled"] - assert doubled["outcome"] == "needs_review", doubled - assert doubled["verdict"] == "REFUTED" - - records = _get(f"{app}/api/orders")["records"] - assert [r["customer"] for r in records] == [ - "Acme Ltd", - "Birch Co", - "Dune LLC", - "Dune LLC", - ] - - # The sign-in values reach no listing, result or program. - program = (tmp_path / "tools" / "add_order" / "program.json").read_text() - for text in [*seen, program]: - assert DEMO_PASSWORD not in text - assert DEMO_USER not in text - assert json.loads(program)["sign_in"] == ["password", "username"] - - -def test_a_taught_write_without_its_sign_in_does_not_run( - app: str, tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: - monkeypatch.delenv("PORTAL_PASSWORD", raising=False) - monkeypatch.setenv("PORTAL_USER", DEMO_USER) - bridge = BrowserTeachBridge( - tmp_path / "tools", - tmp_path / "files", - contracts={"order": _contract(app)}, - secrets={"username": "PORTAL_USER", "password": "PORTAL_PASSWORD"}, - verify_dir=tmp_path / "verify", - ) - with pytest.raises(Exception, match="PORTAL_PASSWORD"): - bridge.dispatch( - "teach_start", + folder = tmp_path / "tools" / "add_order" + folder.mkdir(parents=True) + (folder / bt.PROGRAM_FILE).write_text( + json.dumps( { + "schema": bt.PROGRAM_SCHEMA, "name": "add_order", - "url": f"{app}/login", "task": "Add an order", - "contract": "order", - "inputs": FIRST, - }, + "url": f"{app}/login", + "inputs": ["customer"], + "steps": [], + "writes": [], + "result": {"kind": "write", "contract": "order"}, + } ) + ) + bridge = bt.BrowserTeachBridge( + tmp_path / "tools", tmp_path / "files", contracts={"order": _contract(app)} + ) + result = bridge.dispatch("taught_add_order", {"inputs": {"customer": "Birch"}}) + assert result["outcome"] == "did_not_run", result + assert _orders(app) == before diff --git a/packages/flow/tests/e2e/test_browser_teach_write_guard_e2e.py b/packages/flow/tests/e2e/test_browser_teach_write_guard_e2e.py index 1dcd3f1717..f5250c690e 100644 --- a/packages/flow/tests/e2e/test_browser_teach_write_guard_e2e.py +++ b/packages/flow/tests/e2e/test_browser_teach_write_guard_e2e.py @@ -1,131 +1,37 @@ -"""A taught write sends only its taught requests, and sign-in values stay put. +"""A taught write sends nothing while taught writes are refused. -From the #1589 review, in real headless Chromium against a small app with a -sign-in, a GET search box, a POST order form and a POST "Close account" -button. The server counts what it receives. - -* Teaching: a write the contract doesn't list (Close account) is stopped - before it is sent, and run 1 ends. -* Teaching: a sign-in value can't be typed into the search box (it would go - into an address) or into the order form; a request whose address carries - one, in any encoding, never leaves. -* Calling the tool: a page that adds a field to the order, or a second write, - is stopped before it is sent, and the result is never done. -* No form of the password (raw, percent, form, JSON, base64) appears in any - result. The password has @, a space and &. -* A verified result carries a Seal of its receipt and says the values were - declared by the agent. +Browser teach refuses taught writes until a person's signed teach +authorization is bound to them (docs/TEACH_AUTHORIZATION.md). Against the +real example app: ``teach_start`` with a write contract is refused before a +browser opens, and a write tool already on disk doesn't run. Nothing is +written to the app. """ from __future__ import annotations -import http.server import json -import re -import threading -import urllib.parse +import urllib.request from pathlib import Path -from typing import Iterator +from typing import Any, Iterator import pytest -pytest.importorskip("playwright") - -from openadapt_flow.agent.bridge import BridgeError # noqa: E402 -from openadapt_flow.agent.browser_teach import BrowserTeachBridge # noqa: E402 -from openadapt_flow.verify import WriteContract # noqa: E402 -from openadapt_flow.wire_check import secret_forms # noqa: E402 - -PASSWORD = "p@ss w0rd&x" - - -class _App: - def __init__(self) -> None: - self.orders: list[dict[str, str]] = [] - self.closed = 0 - self.mode = "" - app = self - - class Handler(http.server.BaseHTTPRequestHandler): - def log_message(self, *_a: object) -> None: - pass - - def _send( - self, body: bytes, ctype: str = "text/html", code: int = 200 - ) -> None: - self.send_response(code) - self.send_header("Content-Type", ctype) - self.send_header("Content-Length", str(len(body))) - self.end_headers() - self.wfile.write(body) - - def _page(self, title: str, body: str) -> None: - extra = "" - if app.mode == "extra_field": - extra = """""" - if app.mode == "second_write": - extra = """""" - self._send( - f"{title}{body}{extra}".encode() - ) - - def do_GET(self) -> None: # noqa: N802 - path = urllib.parse.urlsplit(self.path).path - if path == "/login": - self._page( - "Sign in", - '
' - "
", - ) - elif path in ("/app", "/search"): - self._page( - "App", - '
" - '
' - '
", - ) - elif path == "/api/orders": - self._send( - json.dumps({"records": app.orders}).encode(), "application/json" - ) - else: - self._send(b"nf", code=404) +from openadapt_flow.agent import browser_teach as bt +from openadapt_flow.agent.bridge import BridgeError +from openadapt_flow.agent.example_app import serve +from openadapt_flow.verify import WriteContract - def do_POST(self) -> None: # noqa: N802 - length = int(self.headers.get("Content-Length") or 0) - raw = self.rfile.read(length).decode() - if "json" in (self.headers.get("Content-Type") or ""): - form = json.loads(raw) - else: - form = dict(urllib.parse.parse_qsl(raw)) - if self.path == "/orders": - app.orders.append({"customer": form.get("customer", "")}) - elif self.path == "/account/close": - app.closed += 1 - self.send_response(303) - self.send_header("Location", "/app") - self.end_headers() - self.http = http.server.ThreadingHTTPServer(("127.0.0.1", 0), Handler) - threading.Thread(target=self.http.serve_forever, daemon=True).start() - self.base = f"http://127.0.0.1:{self.http.server_address[1]}" +@pytest.fixture +def app() -> Iterator[str]: + base, stop = serve(0) + yield base + stop() -@pytest.fixture -def app() -> Iterator[_App]: - server = _App() - yield server - server.http.shutdown() +def _orders(base: str) -> Any: + with urllib.request.urlopen(f"{base}/api/orders", timeout=5) as response: + return json.loads(response.read())["records"] def _contract(base: str) -> WriteContract: @@ -153,140 +59,41 @@ def _contract(base: str) -> WriteContract: ) -def _ref(listing: str, label: str) -> str: - for line in listing.splitlines(): - found = re.match(r"\[(e\d+)\] \S+ \"" + re.escape(label) + '"', line) - if found: - return found.group(1) - raise AssertionError(f"no {label!r}:\n{listing}") - - -@pytest.fixture -def bridge( - app: _App, tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> BrowserTeachBridge: - monkeypatch.setenv("P_USER", "clerk") - monkeypatch.setenv("P_PW", PASSWORD) - monkeypatch.setenv("OPENADAPT_SEAL_HOME", str(tmp_path / "seal")) - return BrowserTeachBridge( - tmp_path / "tools", - tmp_path / "files", - contracts={"order": _contract(app.base)}, - secrets={"username": "P_USER", "password": "P_PW"}, - verify_dir=tmp_path / "verify", - ) - - -def _start(bridge: BrowserTeachBridge, app: _App) -> tuple[str, list[str]]: - seen: list[str] = [] - started = bridge.dispatch( - "teach_start", - { - "name": "add_order", - "url": f"{app.base}/login", - "task": "Add an order", - "contract": "order", - "inputs": {"customer": "Acme"}, - }, - ) - seen.append(json.dumps(started)) - page = started["page"] - for args in ( - {"action": "type", "ref": _ref(page, "Username"), "input": "username"}, - {"action": "type", "ref": _ref(page, "Password"), "input": "password"}, - ): - result = bridge.dispatch("teach_act", args) - seen.append(json.dumps(result)) - page = result["page"] - result = bridge.dispatch( - "teach_act", {"action": "click", "ref": _ref(page, "Sign in")} +def test_no_taught_write_reaches_the_app(app: str, tmp_path: Path) -> None: + before = _orders(app) + bridge = bt.BrowserTeachBridge( + tmp_path / "tools", tmp_path / "files", contracts={"order": _contract(app)} ) - seen.append(json.dumps(result)) - return result["page"], seen - - -def _assert_no_password(texts: list[str]) -> None: - for text in texts: - for form in secret_forms(PASSWORD): - assert form not in text, form - - -def test_an_undeclared_write_is_never_sent( - bridge: BrowserTeachBridge, app: _App -) -> None: - page, seen = _start(bridge, app) - page = bridge.dispatch( - "teach_act", - {"action": "type", "ref": _ref(page, "Customer"), "input": "customer"}, - )["page"] - page = bridge.dispatch( - "teach_act", {"action": "click", "ref": _ref(page, "Save order")} - )["page"] - with pytest.raises( - BridgeError, match="POST /account/close, which the write contract" - ): + with pytest.raises(BridgeError, match="taught writes are refused"): bridge.dispatch( - "teach_act", {"action": "click", "ref": _ref(page, "Close account")} + "teach_start", + { + "name": "add_order", + "url": f"{app}/login", + "task": "Add an order", + "contract": "order", + "inputs": {"customer": "Acme Ltd"}, + }, ) - assert app.closed == 0 - _assert_no_password(seen) - - -def test_a_sign_in_value_stays_out_of_addresses_and_records( - bridge: BrowserTeachBridge, app: _App -) -> None: - page, seen = _start(bridge, app) - for label in ("Search", "Customer"): - with pytest.raises( - BridgeError, match="sign-in value goes only into" - ) as refused: - bridge.dispatch( - "teach_act", - {"action": "type", "ref": _ref(page, label), "input": "password"}, - ) - seen.append(str(refused.value)) - # Even typed as plain text, a request carrying it never leaves. - page = bridge.dispatch( - "teach_act", {"action": "type", "ref": _ref(page, "Search"), "text": PASSWORD} - )["page"] - with pytest.raises(BridgeError, match="sign-in value in its address") as ended: - bridge.dispatch("teach_act", {"action": "click", "ref": _ref(page, "Find")}) - seen.append(str(ended.value)) - _assert_no_password(seen) - - -@pytest.mark.parametrize("mode", ["extra_field", "second_write"]) -def test_a_taught_write_sends_only_what_it_was_taught( - bridge: BrowserTeachBridge, app: _App, mode: str -) -> None: - page, seen = _start(bridge, app) - page = bridge.dispatch( - "teach_act", - {"action": "type", "ref": _ref(page, "Customer"), "input": "customer"}, - )["page"] - bridge.dispatch("teach_act", {"action": "click", "ref": _ref(page, "Save order")}) - finished = bridge.dispatch("teach_finish", {}) - seen.append(json.dumps(finished)) - assert finished["status"] == "ready", finished - program = json.loads(Path(finished["program"]).read_text()) - assert [w["write"] for w in program["writes"]] == ["POST /login", "POST /orders"] - - good = bridge.dispatch("taught_add_order", {"inputs": {"customer": "Birch"}}) - seen.append(json.dumps(good)) - assert good["outcome"] == "done", good - assert good["values_declared_by"].startswith("agent") - assert good["authorization"] == "unbound" - assert good["seal_path"] and Path(good["seal_path"]).exists() - assert good["seal"] == "sealed" - - app.mode = mode - bad = bridge.dispatch("taught_add_order", {"inputs": {"customer": "Cedar"}}) - seen.append(json.dumps(bad)) - assert bad["outcome"] == "needs_review", bad - assert bad["verified"] is False - assert "it was not sent" in bad["what_happened"] - customers = [o["customer"] for o in app.orders] - assert "ghost" not in customers # the extra write never left the browser - if mode == "extra_field": - assert "Cedar" not in customers # the changed save was stopped - _assert_no_password(seen) + folder = tmp_path / "tools" / "add_order" + folder.mkdir(parents=True) + (folder / bt.PROGRAM_FILE).write_text( + json.dumps( + { + "schema": bt.PROGRAM_SCHEMA, + "name": "add_order", + "task": "Add an order", + "url": f"{app}/login", + "inputs": ["customer"], + "steps": [], + "writes": [], + "result": {"kind": "write", "contract": "order"}, + } + ) + ) + bridge = bt.BrowserTeachBridge( + tmp_path / "tools", tmp_path / "files", contracts={"order": _contract(app)} + ) + result = bridge.dispatch("taught_add_order", {"inputs": {"customer": "Birch"}}) + assert result["outcome"] == "did_not_run", result + assert _orders(app) == before diff --git a/packages/flow/tests/e2e/test_code_action_e2e.py b/packages/flow/tests/e2e/test_code_action_e2e.py index 03283daf64..e90fe04f3e 100644 --- a/packages/flow/tests/e2e/test_code_action_e2e.py +++ b/packages/flow/tests/e2e/test_code_action_e2e.py @@ -30,6 +30,7 @@ from openadapt_flow.compile_agent.harness import Harness, SaveGate from openadapt_flow.mockmed.fault_server import serve from openadapt_flow.tutorial import _http_json, _records +from tests._teach_authorization_support import mockmed_note PROMPT = ( "Open the referral for Jane Sample, start a Triage encounter, and save " @@ -77,6 +78,15 @@ def _compiler(tmp_path: Path, base: str, task_id: str) -> FirstRunCompiler: key_dir=tmp_path / "keys", reset=lambda: _http_json(f"{base}/api/reset", method="POST", body={}), on_warning=lambda text: None, + teach_authorization=mockmed_note( + tmp_path / "keys", + tmp_path / "teach-authorization", + task_id, + base, + note=NOTE, + record={"patient_id": "p1", "type": "Triage"}, + constants=["Jane Sample", "MRN P1"], + ), ) diff --git a/packages/flow/tests/e2e/test_code_agent_default_e2e.py b/packages/flow/tests/e2e/test_code_agent_default_e2e.py index ec8262fd04..605ed04547 100644 --- a/packages/flow/tests/e2e/test_code_agent_default_e2e.py +++ b/packages/flow/tests/e2e/test_code_agent_default_e2e.py @@ -129,7 +129,9 @@ def test_a_dropped_save_is_reported_as_not_saved(mockmed: str, tmp_path: Path) - # -- the MCP teach path ---------------------------------------------------------- -def _teach(tmp_path: Path, base: str, *, hold: bool) -> object: +def _teach( + tmp_path: Path, base: str, *, hold: bool, teach_authorization: object = None +) -> object: from openadapt_flow.agent.teach import TeachBridge, TeachConfig return TeachBridge( @@ -140,6 +142,7 @@ def _teach(tmp_path: Path, base: str, *, hold: bool) -> object: key_dir=tmp_path / "keys", hold=hold, reset=_reset(base), + teach_authorization=teach_authorization, ) ) @@ -214,3 +217,91 @@ def test_teach_run_stops_before_a_save_a_person_must_approve( assert report["status"] == "draft", report finally: teach.close() # type: ignore[attr-defined] + + +# -- teach_run under a person's signed teach authorization ----------------------- +# Real pieces: a real approver roster and signature, the real teach scope on +# run 1 and the real admission gate (docs/TEACH_AUTHORIZATION.md). + + +def _gate(teach: object, trust: object = None) -> dict: + from openadapt_flow.jit import lifecycle + + jit = teach._compiler._jit() # type: ignore[attr-defined] + return lifecycle.teach_authorization_gate( + jit.program_dir, jit.candidate(), key_dir=jit.key_dir, trust=trust + ) + + +def _authority(tmp_path: Path, base: str) -> object: + from tests._teach_authorization_support import mockmed_note + + return mockmed_note( + tmp_path / "keys", + tmp_path / "teach-authorization", + "taught.add_note", + base, + note=NOTE, + record={"patient_id": "p1", "type": "Triage"}, + constants=["Jane Sample", "MRN P1"], + ) + + +def test_a_teach_run_draft_without_a_statement_is_refused_at_admission( + mockmed: str, tmp_path: Path +) -> None: + from openadapt_flow.jit import AdmissionError + + teach = _teach(tmp_path, mockmed, hold=False) + try: + _start(teach) + for script in SCRIPTS[:2]: + out = teach.dispatch( # type: ignore[attr-defined] + "teach_run", {"script": script.replace("{note}", NOTE)} + ) + assert out["status"] == "ok", out + assert teach.dispatch("teach_finish", {})["status"] == "draft" # type: ignore[attr-defined] + with pytest.raises(AdmissionError, match="teach_auth_not_recorded"): + _gate(teach) + finally: + teach.close() # type: ignore[attr-defined] + + +def test_a_teach_run_draft_under_a_statement_is_bound_and_admissible( + mockmed: str, tmp_path: Path +) -> None: + authority = _authority(tmp_path, mockmed) + teach = _teach(tmp_path, mockmed, hold=False, teach_authorization=authority) + try: + started = _start(teach) + assert started["started"], started + for script in SCRIPTS[:2]: + out = teach.dispatch( # type: ignore[attr-defined] + "teach_run", {"script": script.replace("{note}", NOTE)} + ) + assert out["status"] == "ok", out + report = teach.dispatch("teach_finish", {}) # type: ignore[attr-defined] + assert report["status"] == "draft", report + recorded = _gate(teach, trust=authority) + assert recorded["task_id"] == "taught.add_note" + assert recorded["authorizer"] == "teach-lead" + finally: + teach.close() # type: ignore[attr-defined] + + +def test_teach_run_cannot_type_what_the_statement_does_not_allow( + mockmed: str, tmp_path: Path +) -> None: + teach = _teach( + tmp_path, mockmed, hold=False, teach_authorization=_authority(tmp_path, mockmed) + ) + try: + _start(teach) + out = teach.dispatch("teach_run", {"script": SCRIPTS[0]}) # type: ignore[attr-defined] + assert out["status"] == "ok", out + injected = SCRIPTS[1].replace("{note}", "Cleared for discharge") + out = teach.dispatch("teach_run", {"script": injected}) # type: ignore[attr-defined] + assert out["status"] in ("refused", "error"), out + assert _records(mockmed) == [] # nothing outside the statement was saved + finally: + teach.close() # type: ignore[attr-defined] diff --git a/packages/flow/tests/e2e/test_compile_agent_e2e.py b/packages/flow/tests/e2e/test_compile_agent_e2e.py index 7133b6effd..8ac55378fd 100644 --- a/packages/flow/tests/e2e/test_compile_agent_e2e.py +++ b/packages/flow/tests/e2e/test_compile_agent_e2e.py @@ -38,6 +38,7 @@ from openadapt_flow.jit.keys import CustomerModelKey from openadapt_flow.mockmed.fault_server import serve from openadapt_flow.tutorial import _http_json, _records +from tests._teach_authorization_support import mockmed_note PROMPT = ( "Open the referral for Jane Sample, start a Triage encounter, and save " @@ -204,6 +205,16 @@ def _common(tmp_path: Path, base: str) -> dict[str, Any]: model_key=TEST_KEY, reset=_reset(base), on_warning=lambda text: None, + # A person signed what run 1 may do. + teach_authorization=mockmed_note( + tmp_path / "keys", + tmp_path / "teach-authorization", + "mockmed.compile_agent_note", + base, + note=NOTE, + record=EXPECT, + constants=ON_SCREEN, + ), ) @@ -388,6 +399,18 @@ def reset_first(self: FirstRunCompiler) -> None: "--canary-param", "note=Canary test note", "--json", + "--teach-authorization", + str( + mockmed_note( + key_dir, + tmp_path / "teach-authorization", + "mockmed.compile_agent_cli", + mockmed, + note=NOTE, + record={"patient_id": "p1"}, + constants=["Jane Sample", "MRN P1"], + ).statement + ), ] # No terminal: run 1 passes, and the checks wait for a person. args = build_parser().parse_args(argv) diff --git a/packages/flow/tests/e2e/test_jit_cua_e2e.py b/packages/flow/tests/e2e/test_jit_cua_e2e.py index 1b5490fbcf..7c8c9b08df 100644 --- a/packages/flow/tests/e2e/test_jit_cua_e2e.py +++ b/packages/flow/tests/e2e/test_jit_cua_e2e.py @@ -35,6 +35,7 @@ from openadapt_flow.mockmed.fault_server import serve from openadapt_flow.runtime.effects import RestRecordVerifier from openadapt_flow.tutorial import _http_json, _records +from tests._teach_authorization_support import mockmed_note FIXTURES = Path(__file__).resolve().parents[1] / "fixtures" / "jit" TASK = TaskSignature( @@ -82,6 +83,15 @@ def _jit(base: str, agent: Any, cache: Path, *, fault: str = "ok", on_halt: Any on_halt=on_halt, key_dir=cache.parent / "admission-keys", on_warning=lambda text: None, + # A person signed what the exploring agent may do. + teach_authorization=mockmed_note( + cache.parent / "admission-keys", + cache.parent / "teach-authorization", + TASK.task_id, + base, + note=DEMO_NOTE, + record={"type": "Triage"}, + ), ) diff --git a/packages/flow/tests/e2e/test_mcp_teach_e2e.py b/packages/flow/tests/e2e/test_mcp_teach_e2e.py index c83fd278c7..d4f1e0f16d 100644 --- a/packages/flow/tests/e2e/test_mcp_teach_e2e.py +++ b/packages/flow/tests/e2e/test_mcp_teach_e2e.py @@ -46,6 +46,7 @@ from openadapt_flow.agent.teach import TeachBridge, TeachConfig # noqa: E402 from openadapt_flow.mockmed.fault_server import serve # noqa: E402 from openadapt_flow.tutorial import _http_json, _records # noqa: E402 +from tests._teach_authorization_support import mockmed_note TASK = ( "Open the referral for Jane Sample, start a Triage encounter, and save " @@ -139,6 +140,16 @@ def _server( key_dir=tmp_path / "keys", admitter=admitter, reset=_reset(base), + # The operator's signed teach authorization: what run 1 may do. + teach_authorization=mockmed_note( + tmp_path / "keys", + tmp_path / "teach-authorization", + "taught.triage_note", + base, + note=NOTE, + record={"patient_id": "p1", "type": "Triage"}, + constants=["Jane Sample", "MRN P1"], + ), ), workflows=workflows, ) diff --git a/packages/flow/tests/e2e/test_skill_cache_e2e.py b/packages/flow/tests/e2e/test_skill_cache_e2e.py index d277c72439..f4afc7da10 100644 --- a/packages/flow/tests/e2e/test_skill_cache_e2e.py +++ b/packages/flow/tests/e2e/test_skill_cache_e2e.py @@ -41,6 +41,7 @@ from openadapt_flow.jit.keys import CustomerModelKey # noqa: E402 from openadapt_flow.mockmed.fault_server import serve # noqa: E402 from openadapt_flow.tutorial import _http_json, _records # noqa: E402 +from tests._teach_authorization_support import mockmed_note PROMPT = ( "Open the referral for Jane Sample, start a Triage encounter, and save " @@ -195,11 +196,30 @@ def offer(proposal: dict) -> str: ) +def _authorize( + cache: SkillCache, prompt: str, note: str, patient: str, mrn: str, pid: str +) -> None: + """A person signs what run 1 of this prompt may do (its own task id).""" + + key_dir = Path(cache.common["key_dir"]) + task_id = cache.task_id(prompt, {"note": note}) + cache.common["teach_authorization"] = mockmed_note( + key_dir, + key_dir.parent / f"teach-authorization-{task_id}", + task_id, + cache.common["url"], + note=note, + record={"patient_id": pid, "type": "Triage"}, + constants=[patient, mrn], + ) + + def test_run_1_is_compiled_and_run_2_calls_the_skill( mockmed: str, tmp_path: Path, monkeypatch: pytest.MonkeyPatch ) -> None: offered: list = [] cache = _cache(tmp_path, mockmed, monkeypatch, offered) + _authorize(cache, PROMPT, NOTE, "Jane Sample", "MRN P1", "p1") chat = ScriptedChat(patient="Jane Sample", note=NOTE) first = cache.run(PROMPT, {"note": NOTE}, _agent(chat)) assert first.path == "agent" and first.status == "admitted", first.detail @@ -227,6 +247,7 @@ def test_a_wrong_patient_is_refused_and_not_cached( _http_json(f"{mockmed}/api/reset", method="POST", body={}) offered: list = [] cache = _cache(tmp_path, mockmed, monkeypatch, offered) + _authorize(cache, PROMPT, NOTE, "Jane Sample", "MRN P1", "p1") chat = ScriptedChat(patient="Alex Testcase", note=NOTE) first = cache.run(PROMPT, {"note": NOTE}, _agent(chat)) assert first.path == "agent" and first.status == "refused_wrong_record" @@ -245,7 +266,7 @@ def _for_alex(jane: SkillCache) -> SkillCache: expect={"patient_id": "p2", "type": "Triage"}, on_screen=["Alex Testcase", "MRN P2"], ) - return SkillCache( + alex = SkillCache( **common, offer=jane.offer, admitter=jane.admitter, @@ -253,6 +274,8 @@ def _for_alex(jane: SkillCache) -> SkillCache: drop_fields=jane.drop_fields, model_key=jane.model_key, ) + _authorize(alex, P_ALEX, "Alex note", "Alex Testcase", "MRN P2", "p2") + return alex def test_a_skill_never_serves_another_record( @@ -264,6 +287,7 @@ def test_a_skill_never_serves_another_record( reset() jane = _cache(tmp_path, mockmed, monkeypatch, []) jane.common.pop("task_id") + _authorize(jane, P_JANE, "Jane note", "Jane Sample", "MRN P1", "p1") first = jane.run( P_JANE, {"note": "Jane note"}, @@ -338,6 +362,7 @@ def test_another_users_write_elsewhere_does_not_fail_the_skill( _http_json(f"{mockmed}/api/reset", method="POST", body={}) cache = _cache(tmp_path, mockmed, monkeypatch, []) + _authorize(cache, PROMPT, NOTE, "Jane Sample", "MRN P1", "p1") first = cache.run( PROMPT, {"note": NOTE}, _agent(ScriptedChat(patient="Jane Sample", note=NOTE)) ) diff --git a/packages/flow/tests/test_agent_teach_branch.py b/packages/flow/tests/test_agent_teach_branch.py index 9c763bd7ff..c726d8aa36 100644 --- a/packages/flow/tests/test_agent_teach_branch.py +++ b/packages/flow/tests/test_agent_teach_branch.py @@ -57,8 +57,14 @@ def _examples(tmp_path: Path, titles): def test_two_demonstrations_become_one_approved_branch(tmp_path): bundle, pairs = _examples(tmp_path, TITLES) + from tests._teach_authorization_support import repair_authorization + + authorized = repair_authorization(tmp_path / "store", tmp_path / "keys") result = teach_branch_from_calling_agent( - pairs, tmp_path / "proposed", bundle=bundle, provenance=_provenance() + pairs, + tmp_path / "proposed", + bundle=bundle, + provenance=_provenance(teach_authorization=authorized), ) candidate = result.candidate assert candidate.state == "candidate" diff --git a/packages/flow/tests/test_agent_teach_repair.py b/packages/flow/tests/test_agent_teach_repair.py index 56dc344248..72d5702b23 100644 --- a/packages/flow/tests/test_agent_teach_repair.py +++ b/packages/flow/tests/test_agent_teach_repair.py @@ -29,6 +29,7 @@ from openadapt_flow.repair.dialog_identity import enclosing_dialog_region from openadapt_flow.repair.lifecycle import ModelActuationError, RepairLifecycleError from openadapt_flow.runtime.replayer import Replayer +from tests._teach_authorization_support import repair_authorization AGENT_ID = "mcp-client:test-agent" PERSON = Actor(kind="human", id="reviewer@example.org") @@ -86,7 +87,12 @@ def taught(halted, tmp_path: Path): _agent_recording(tmp_path / "agent_recording"), tmp_path / "proposed", bundle=bundle, - provenance=_provenance(), + # A person signed that a calling agent may repair this workflow. + provenance=_provenance( + teach_authorization=repair_authorization( + tmp_path / "store", tmp_path / "keys" + ) + ), ) return bundle, run_dir, result @@ -268,3 +274,64 @@ def test_allowed_screen_provenance_modes() -> None: ) assert raw.screens_scrubbed is False assert _provenance(screens_shared=0, scrub_evidence=None).screens_shared == 0 + + +def _through_campaigns(store: RepairStore, candidate) -> str: + candidate = store.add_candidate(candidate) + store.review(candidate.candidate_id, PERSON) + for kind in ("replay", "fault"): + campaign = _run_campaign(store.load_candidate(candidate.candidate_id), kind) + assert campaign.passed, [c.detail for c in campaign.cases if not c.passed] + store.record_campaign(candidate.candidate_id, campaign, AUTOMATION) + return candidate.candidate_id + + +def test_a_calling_agent_fix_without_a_teach_authorization_is_not_approved( + halted, tmp_path: Path +) -> None: + bundle, run_dir = halted + result = teach_from_calling_agent( + run_dir, + _agent_recording(tmp_path / "agent_recording"), + tmp_path / "proposed", + bundle=bundle, + provenance=_provenance(), + ) + store = RepairStore(tmp_path / "store") + candidate_id = _through_campaigns(store, result.candidate) + with pytest.raises(RepairLifecycleError, match="teach_auth_not_recorded"): + store.approve(candidate_id, PERSON, non_interactive=True) + + +def test_a_calling_agent_fix_after_the_authorizer_was_removed_is_not_approved( + taught, tmp_path: Path +) -> None: + from tests import _teach_authorization_support as support + + _, _, result = taught + store = RepairStore(tmp_path / "store") + candidate_id = _through_campaigns(store, result.candidate) + keys = tmp_path / "keys" + support.roster_root(keys).remove(support.authorizer(keys)) + with pytest.raises(RepairLifecycleError, match="teach_auth_unknown_signer"): + store.approve(candidate_id, PERSON, non_interactive=True) + + +def test_a_teach_only_authorization_does_not_cover_a_repair( + halted, tmp_path: Path +) -> None: + bundle, run_dir = halted + record = repair_authorization( + tmp_path / "store", tmp_path / "keys", purposes=("teach",) + ) + result = teach_from_calling_agent( + run_dir, + _agent_recording(tmp_path / "agent_recording"), + tmp_path / "proposed", + bundle=bundle, + provenance=_provenance(teach_authorization=record), + ) + store = RepairStore(tmp_path / "store") + candidate_id = _through_campaigns(store, result.candidate) + with pytest.raises(RepairLifecycleError, match="teach_auth_purpose"): + store.approve(candidate_id, PERSON, non_interactive=True) diff --git a/packages/flow/tests/test_browser_teach.py b/packages/flow/tests/test_browser_teach.py index d05b0b6b1a..3850dcd6e0 100644 --- a/packages/flow/tests/test_browser_teach.py +++ b/packages/flow/tests/test_browser_teach.py @@ -309,3 +309,37 @@ def check(): assert page.evaluate("window.hit") == 1 _on(browser, check) + + +def test_taught_writes_are_refused(tmp_path: Path) -> None: + """No person's teach authorization binds a write taught in this lane.""" + + bridge = bt.BrowserTeachBridge(tmp_path, tmp_path / "files") + with pytest.raises(BridgeError, match="taught writes are refused"): + bridge.dispatch( + "teach_start", + { + "name": "file_order", + "task": "File the order", + "url": "http://127.0.0.1:1/", + "inputs": {"item": "Widget"}, + "contract": "order", + }, + ) + folder = tmp_path / "file_order" + folder.mkdir() + (folder / bt.PROGRAM_FILE).write_text( + json.dumps( + { + "schema": bt.PROGRAM_SCHEMA, + "name": "file_order", + "task": "File the order", + "inputs": ["item"], + "result": {"kind": "write", "contract": "order"}, + } + ) + ) + bridge = bt.BrowserTeachBridge(tmp_path, tmp_path / "files") + result = bridge.dispatch("taught_file_order", {"inputs": {"item": "Widget"}}) + assert result["outcome"] == "did_not_run" + assert "nothing was written" in result["what_happened"] diff --git a/packages/flow/tests/test_jit_lifecycle.py b/packages/flow/tests/test_jit_lifecycle.py index 5bbe22674f..e5bfe46ea2 100644 --- a/packages/flow/tests/test_jit_lifecycle.py +++ b/packages/flow/tests/test_jit_lifecycle.py @@ -40,6 +40,7 @@ ReadbackSpec, ) from openadapt_flow.verification import VerificationTier +from tests._teach_authorization_support import draft_fields TASK = TaskSignature("t.add_note", "Add a note", {}) REVIEWER = "Dana Reviewer" @@ -106,6 +107,7 @@ def _jit( task: TaskSignature = TASK, require_distinct_reviewer_and_admitter: bool = False, agent_principal_id: Any = "cua-bot-7", + authorized: bool = True, ) -> Any: jit = compile_agent( agent or NeverCalled(), @@ -145,6 +147,8 @@ def _jit( "policy": "clinical-write", "mine_effects": True, "compiled_recording": str(program / "recording"), + # A person signed what the agent could do (admission needs it). + **(draft_fields(program, jit.key_dir, task.task_id) if authorized else {}), }, key_dir=jit.key_dir, ) diff --git a/packages/flow/tests/test_teach_authorization.py b/packages/flow/tests/test_teach_authorization.py new file mode 100644 index 0000000000..2536cfe5c5 --- /dev/null +++ b/packages/flow/tests/test_teach_authorization.py @@ -0,0 +1,387 @@ +"""Teach authorizations: a person signs the scope a teaching agent gets. + +The authorizer is a real approver key on a real signed approver roster +(``LocalAuthorizer`` holds the key in memory); verification is the approver +verifier's own. Nothing here contacts the network. All data is synthetic. +""" + +from __future__ import annotations + +import json +from datetime import datetime, timedelta, timezone +from pathlib import Path +from typing import Any + +import pytest + +from openadapt_flow import teach_authorization as ta +from openadapt_flow.jit.adapters import Op +from openadapt_flow.mockmed.teach_scope import ( + EncounterWrites, + encounter_authorization_spec, + encounter_scope, +) +from openadapt_flow.teach_authorization_fixture import LocalAuthorizer +from openadapt_flow.teach_scope import ScopeViolation, TeachGuard, TeachScope + +BASE = "http://127.0.0.1:8765" +NOTE = "Synthetic follow-up in two weeks" + + +@pytest.fixture(autouse=True) +def _no_env(monkeypatch): + monkeypatch.delenv("OPENADAPT_APPROVER_ROOT", raising=False) + + +@pytest.fixture +def person(tmp_path) -> LocalAuthorizer: + return LocalAuthorizer(tmp_path / "keys", "front-desk") + + +def _issue(tmp_path: Path, person: LocalAuthorizer, **kw: Any) -> Path: + spec = encounter_authorization_spec(BASE, patient_id="P1", note=NOTE, **kw) + return person.issue(tmp_path / "auth", spec) + + +def _require(person: LocalAuthorizer, statement: Path, **kw: Any) -> ta.Verification: + kw.setdefault("purpose", "teach") + return ta.require(statement, key_dir=person.key_dir, root=person.root, **kw) + + +def _code(fn, *args, **kw) -> str: + with pytest.raises(ta.TeachAuthorizationError) as err: + fn(*args, **kw) + return err.value.code + + +def test_a_signed_authorization_binds_the_scope(tmp_path, person): + statement = _issue(tmp_path, person) + result = _require(person, statement) + assert result.signer == "front-desk" + scope = ta.bind(result, {"note": NOTE}) + assert scope.authorized + assert dict(scope.record) == {"patient_id": "P1"} + assert dict(scope.fields) == {"type": "Triage", "note": NOTE} + assert scope.origins == (BASE,) + assert isinstance(scope.write_adapter, EncounterWrites) + assert "delete_request" in scope.forbidden_actions + assert scope.authorization["sha256"] == result.sha256 + # The report names the authorization but holds no values. + report = json.dumps(scope.to_dict()) + assert NOTE not in report and result.sha256 in report + + +def test_the_roster_root_must_be_pinned(tmp_path, person): + statement = _issue(tmp_path, person) + assert ( + _code(ta.require, statement, purpose="teach", key_dir=person.key_dir, root=None) + == "teach_auth_identity_unverified" + ) + other = LocalAuthorizer(tmp_path / "other-keys", "front-desk") + assert ( + _code( + ta.require, + statement, + purpose="teach", + key_dir=person.key_dir, + root=other.root, + ) + == "teach_auth_identity_unverified" + ) + # A key directory with no roster vouches for nobody. + assert ( + _code( + ta.require, + statement, + purpose="teach", + key_dir=tmp_path / "empty", + root=person.root, + ) + == "teach_auth_identity_unverified" + ) + + +def test_an_unknown_key_refuses(tmp_path, person): + stranger = LocalAuthorizer(person.key_dir, "night-desk", register=False) + forged = stranger.issue( + tmp_path / "forged", + encounter_authorization_spec(BASE, patient_id="P2", note=NOTE), + ) + assert _code(_require, person, forged) == "teach_auth_unknown_signer" + + +def test_a_key_signing_as_another_approver_refuses(tmp_path, person): + # The agent makes its own key and signs in the root approver's name. + agent = LocalAuthorizer(person.key_dir, "agent-key", register=False) + forged = agent.issue( + tmp_path / "forged", + encounter_authorization_spec(BASE, patient_id="P2", note=NOTE), + authorizer="front-desk", + ) + assert _code(_require, person, forged) == "teach_auth_bad_signature" + + +def test_a_revoked_key_refuses(tmp_path, person): + nurse = LocalAuthorizer(person.key_dir, "night-desk", added_by=person) + statement = nurse.issue( + tmp_path / "auth", + encounter_authorization_spec(BASE, patient_id="P1", note=NOTE), + ) + recorded = ta.record(_require(person, statement)) + person.remove(nurse) + assert _code(_require, person, statement) == "teach_auth_unknown_signer" + # Admission checks the roster as it is now, so revoking after teaching refuses. + assert ( + _code( + ta.check_admission, + recorded, + statement, + key_dir=person.key_dir, + root=person.root, + ) + == "teach_auth_unknown_signer" + ) + + +def test_an_old_roster_put_back_refuses(tmp_path, person): + roster_file = person.key_dir / "approvers.json" + before = roster_file.read_bytes() + nurse = LocalAuthorizer(person.key_dir, "night-desk", added_by=person) + person.remove(nurse) + statement = _issue(tmp_path, person) + roster_file.write_bytes(before) # the head sidecar still names the newer roster + assert _code(_require, person, statement) == "teach_auth_identity_unverified" + + +def test_an_attestation_labelled_for_the_portal_is_refused(tmp_path): + portal = LocalAuthorizer(tmp_path / "keys", "portal", method="portal_key") + statement = portal.issue( + tmp_path / "auth", + encounter_authorization_spec(BASE, patient_id="P1", note=NOTE), + ) + assert _code(_require, portal, statement) == "teach_auth_method" + + +def test_an_edited_statement_fails(tmp_path, person): + statement = _issue(tmp_path, person) + data = json.loads(statement.read_text()) + data["writes"]["identity"]["patient_id"]["value"] = "P2" + statement.write_text(json.dumps(data)) + assert _code(_require, person, statement) == "teach_auth_bad_signature" + + +def test_an_unsigned_statement_fails(tmp_path, person): + statement = _issue(tmp_path, person) + statement.with_name(ta.SIGNATURE_FILE).unlink() + assert _code(_require, person, statement) == "teach_auth_unsigned" + + +def test_sign_with_a_software_key_file(tmp_path, monkeypatch): + """No Secure Enclave: an encrypted key file, added to the roster, signs.""" + + from openadapt_flow.approval_mode import approver_roster as roster + from openadapt_flow.approval_mode.human_attestation import load_signer + from openadapt_flow.jit.authority import ( + KEY_PASSPHRASE_ENV, + generate_encrypted_key, + load_encrypted_key, + ) + + env = {KEY_PASSPHRASE_ENV: "synthetic-passphrase"} + keys = tmp_path / "keys" + root_path, mine_path = tmp_path / "root.pem", tmp_path / "mine.pem" + root_public = generate_encrypted_key(root_path, env=env) + root_key = load_encrypted_key(root_path, env=env) + roster.init_root( + keys, + principal="admin", + public_key=root_public, + sign=roster.file_key_signer(root_key), + ) + mine = generate_encrypted_key(mine_path, env=env) + roster.add_approver( + keys, + root_public, + principal="clinic-lead", + public_key=mine, + signed_by="admin", + sign=roster.file_key_signer(root_key), + ) + spec = encounter_authorization_spec(BASE, patient_id="P1", note=NOTE) + statement = ta.create( + tmp_path / "auth", spec, authorizer="clinic-lead", authorizer_public_key=mine + ) + # A key file for another approver can't sign it. + admin = load_signer(root_path, keys, method="person_key", env=env, root=root_public) + assert _code(ta.sign, statement, admin) == "teach_auth_signer_mismatch" + signer = load_signer( + mine_path, keys, method="person_key", env=env, root=root_public + ) + ta.sign(statement, signer) + assert _code(ta.sign, statement, signer) == "teach_auth_already_signed" + result = ta.require(statement, purpose="teach", key_dir=keys, root=root_public) + assert ( + result.ok and result.signer == "clinic-lead" and result.method == "person_key" + ) + + +def test_purpose_task_and_expiry(tmp_path, person): + statement = _issue(tmp_path, person, purposes=("teach",)) + assert _code(_require, person, statement, purpose="repair") == "teach_auth_purpose" + assert ( + _code(_require, person, statement, task_id="other") + == "teach_auth_task_mismatch" + ) + later = datetime.now(timezone.utc) + timedelta(days=2) + assert _code(_require, person, statement, now=later) == "teach_auth_expired" + + +def test_digest_bound_inputs(tmp_path, person): + spec = encounter_authorization_spec( + BASE, patient_id="P1", note_sha256=ta.value_sha256(NOTE) + ) + statement = person.issue(tmp_path / "auth", spec) + assert NOTE not in statement.read_text() + result = _require(person, statement) + assert dict(ta.bind(result, {"note": NOTE}).fields)["note"] == NOTE + for params, code in ( + ({}, "teach_auth_input_missing"), + ({"note": "cleared for discharge"}, "teach_auth_input_mismatch"), + ({"note": NOTE, "extra": "x"}, "teach_auth_input_not_declared"), + ): + assert _code(ta.bind, result, params) == code + + +def test_a_scope_from_flags_may_narrow_but_not_widen(tmp_path, person): + statement = _issue(tmp_path, person) + authorized = ta.bind(_require(person, statement)) + ta.check_within(encounter_scope(BASE, patient_id="P1", note=NOTE), authorized) + for widened in ( + encounter_scope(BASE, patient_id="P2", note=NOTE), + encounter_scope(BASE, patient_id="P1", note="cleared for discharge"), + encounter_scope(BASE, patient_id="P1", note=NOTE, constants=("x",)), + TeachScope( + task_id="mockmed.add_note", + origins=(BASE, "http://127.0.0.1:9999"), + record={"patient_id": "P1"}, + fields={"type": "Triage", "note": NOTE}, + ), + ): + assert _code(ta.check_within, widened, authorized) == "teach_auth_scope_widened" + + +def test_admission_reverifies_the_recorded_digest(tmp_path, person): + statement = _issue(tmp_path, person) + recorded = ta.record(_require(person, statement)) + copy = ta.copy_into(statement, tmp_path / "draft") + trust = {"key_dir": person.key_dir, "root": person.root} + assert ta.check_admission(recorded, copy, **trust).ok + assert _code(ta.check_admission, None, copy, **trust) == "teach_auth_not_recorded" + assert _code(ta.check_admission, recorded, None, **trust) == "teach_auth_missing" + assert ( + _code(ta.check_admission, recorded, copy, key_dir=person.key_dir, root=None) + == "teach_auth_identity_unverified" + ) + other = _issue(tmp_path / "second", person) + assert ( + _code(ta.check_admission, recorded, other, **trust) + == "teach_auth_digest_mismatch" + ) + assert ( + _code( + ta.check_admission, recorded, copy, purpose="repair", task_id="x", **trust + ) + == "teach_auth_task_mismatch" + ) + assert ( + _code(ta.check_admission, recorded, copy, not_signed_by="front-desk", **trust) + == "teach_auth_agent_signer" + ) + # A different statement can't replace the kept copy. + assert ( + _code(ta.copy_into, other, tmp_path / "draft") == "teach_auth_digest_mismatch" + ) + + +def test_statement_rules(): + base = encounter_authorization_spec(BASE, patient_id="P1", note=NOTE) + fields = { + "schema": ta.SCHEMA, + "authorizer": "front-desk", + "authorizer_public_key": "a" * 64, + "issued_at": "2026-10-20T00:00:00Z", + "nonce": "a" * 64, + } + ta.TeachAuthorizationV1.model_validate({**base, **fields}) + bad = [ + {**base, "origins": ["http://127.0.0.1:8765/path"]}, + {**base, "inputs": {"note": {"value": NOTE, "sha256": "a" * 64}}}, + {**base, "writes": {**base["writes"], "identity": {}}}, + {**base, "writes": {**base["writes"], "fields": {"note": {"input": "nope"}}}}, + {**base, "forbidden": {"actions": ["teleport"]}}, + {**base, "purposes": ["run"]}, + {**base, "constants": [{"sha256": "a" * 64}]}, + {**base, "surprise": True}, + {**base, "authorizer_public_key": "github:someone"}, + ] + for spec in bad: + with pytest.raises(ValueError): + ta.TeachAuthorizationV1.model_validate({**fields, **spec}) + + +# -- the guard enforces the forbidden list ----------------------------------------- + + +class _Page: + """Enough of a Playwright page for the guard's probes.""" + + url = BASE + "/" + + def __init__(self, label: str = "", link: Any = None) -> None: + self.label, self.link = label, link + + def evaluate(self, script: str, args: Any) -> Any: + return self.label if "aria-label" in script else self.link + + +def _guard(tmp_path: Path, person: LocalAuthorizer) -> TeachGuard: + statement = _issue(tmp_path, person) + return TeachGuard(ta.bind(_require(person, statement))) + + +def test_forbidden_controls_and_actions(tmp_path, person): + guard = _guard(tmp_path, person) + with pytest.raises(ScopeViolation) as err: + guard.check_ops([Op("click", x=1, y=1)], page=_Page(" delete ENCOUNTER ")) + assert err.value.code == "control_forbidden" + guard.check_ops([Op("click", x=1, y=1)], page=_Page("Save encounter")) + with pytest.raises(ScopeViolation) as err: + guard.check_ops([Op("drag", x=1, y=1, end_x=2, end_y=2)]) + assert err.value.code == "forbidden_action" + assert ( + guard.scope.request_refusal("DELETE", BASE + "/api/encounter/1", None) + == "forbidden_action" + ) + + +def test_a_broken_label_probe_refuses_the_click(tmp_path, person): + class _Broken(_Page): + def evaluate(self, script: str, args: Any) -> Any: + if "aria-label" in script: + raise RuntimeError("the page overrode elementFromPoint") + return None + + guard = _guard(tmp_path, person) + with pytest.raises(ScopeViolation) as err: + guard.check_ops([Op("click", x=1, y=1)], page=_Broken()) + assert err.value.code == "control_forbidden" + # A control with no label is not a forbidden one. + guard.check_ops([Op("click", x=1, y=1)], page=_Page("")) + + +def test_render_shows_what_is_signed(tmp_path, person): + statement = _issue(tmp_path, person) + text = ta.render(_require(person, statement)) + assert "patient_id = 'P1'" in text + assert "Delete encounter" in text + assert BASE in text diff --git a/packages/flow/tests/test_teach_authorization_admission.py b/packages/flow/tests/test_teach_authorization_admission.py new file mode 100644 index 0000000000..cf9e2f47fc --- /dev/null +++ b/packages/flow/tests/test_teach_authorization_admission.py @@ -0,0 +1,209 @@ +"""Admission of an agent-taught JIT draft needs a person's signed teach authorization. + +Real pieces throughout: a real approver roster in the key directory, real +approver signatures, the real verifier (the approver verifier's own), the +real recorder signature and the real admission code. Only the canary's run +is a stub report, as in tests/test_jit_lifecycle.py. All data is synthetic. +""" + +from __future__ import annotations + +import json +import shutil +from pathlib import Path + +import pytest + +from openadapt_flow import teach_authorization as ta +from openadapt_flow.approval_mode.approver_roster import ROOT_ENV +from openadapt_flow.jit import AdmissionError, lifecycle +from openadapt_flow.jit.lifecycle import admit_repair +from tests import _teach_authorization_support as support +from tests.test_jit_lifecycle import ( # noqa: F401 (autouse fixture) + REVIEWER, + _ci, + _jit, + _policy_passes, + _report, + _review, + _stub_runs, +) + + +def _ready(tmp_path: Path, monkeypatch: pytest.MonkeyPatch, **kw): + jit = _jit(tmp_path, **kw) + _review(jit) + _stub_runs(jit, monkeypatch, _report()) + jit.canary(principal_id=REVIEWER) + return jit + + +def _refusal(jit, monkeypatch) -> str: + with pytest.raises(AdmissionError) as err: + jit.admit_with_key(_ci(jit, monkeypatch)) + return str(err.value) + + +def test_an_authorized_draft_admits_and_keeps_the_statement(tmp_path, monkeypatch): + jit = _ready(tmp_path, monkeypatch) + decision = jit.admit_with_key(_ci(jit, monkeypatch)) + assert decision["outcome"] == "admitted" + manifest = decision["manifest"] + assert manifest["teach_authorization"]["authorizer"] == support.AUTHORIZER + kept = jit.bundle_dir / lifecycle.TEACH_AUTHORIZATION_DIR / ta.STATEMENT_FILE + assert kept.is_file() + assert lifecycle.admission_status(jit.program_dir, key_dir=jit.key_dir) + + +def test_a_draft_without_an_authorization_is_refused(tmp_path, monkeypatch): + jit = _ready(tmp_path, monkeypatch, authorized=False) + assert "teach_auth_not_recorded" in _refusal(jit, monkeypatch) + status = lifecycle.admission_status(jit.program_dir, key_dir=jit.key_dir) + assert status["requirements"]["teach_authorization"]["met"] is False + + +def test_an_authorizer_removed_after_teaching_is_refused(tmp_path, monkeypatch): + jit = _ready(tmp_path, monkeypatch) + root = support.roster_root(jit.key_dir) + root.remove(support.authorizer(jit.key_dir)) + assert "teach_auth_unknown_signer" in _refusal(jit, monkeypatch) + + +def test_an_unpinned_approver_root_is_refused(tmp_path, monkeypatch): + jit = _ready(tmp_path, monkeypatch) + monkeypatch.delenv(ROOT_ENV) + assert "teach_auth_identity_unverified" in _refusal(jit, monkeypatch) + + +def test_another_roster_root_is_refused(tmp_path, monkeypatch): + jit = _ready(tmp_path, monkeypatch) + from openadapt_flow.teach_authorization_fixture import LocalAuthorizer + + monkeypatch.setenv(ROOT_ENV, LocalAuthorizer(tmp_path / "elsewhere").root) + assert "teach_auth_identity_unverified" in _refusal(jit, monkeypatch) + + +def test_a_replaced_statement_is_refused(tmp_path, monkeypatch): + jit = _ready(tmp_path, monkeypatch) + kept = jit.program_dir / lifecycle.TEACH_AUTHORIZATION_DIR + other = support.issue(jit.key_dir, tmp_path / "other", "t.add_note") + for name in (ta.STATEMENT_FILE, ta.SIGNATURE_FILE): + shutil.copy(other.with_name(name), kept / name) + assert "teach_auth_digest_mismatch" in _refusal(jit, monkeypatch) + + +def test_an_edited_statement_is_refused(tmp_path, monkeypatch): + jit = _ready(tmp_path, monkeypatch) + path = jit.program_dir / lifecycle.TEACH_AUTHORIZATION_DIR / ta.STATEMENT_FILE + data = json.loads(path.read_text()) + data["origins"] = ["http://127.0.0.1:9", "http://127.0.0.1:10"] + path.write_text(json.dumps(data)) + assert "teach_auth_bad_signature" in _refusal(jit, monkeypatch) + + +def test_stripping_the_authorization_from_the_candidate_is_refused( + tmp_path, monkeypatch +): + jit = _ready(tmp_path, monkeypatch) + path = jit.program_dir / "candidate.json" + candidate = json.loads(path.read_text()) + candidate.pop("teach_authorization") + candidate["demonstrated_by"] = "person" + path.write_text(json.dumps(candidate)) + # The recorder block is signed; the gate reads it, not the editable fields. + assert "changed after it was recorded" in _refusal(jit, monkeypatch) + + +def test_an_authorization_for_another_task_is_refused(tmp_path, monkeypatch): + jit = _jit(tmp_path, authorized=False) + program = jit.program_dir + candidate = json.loads((program / "candidate.json").read_text()) + candidate.pop("recorder") + candidate.update(support.draft_fields(program, jit.key_dir, "t.other_task")) + candidate = lifecycle.sign_recorder(program, candidate, key_dir=jit.key_dir) + (program / "candidate.json").write_text(json.dumps(candidate)) + _review(jit) + _stub_runs(jit, monkeypatch, _report()) + jit.canary(principal_id=REVIEWER) + assert "teach_auth_task_mismatch" in _refusal(jit, monkeypatch) + + +def test_a_capture_not_bound_to_the_authorization_is_refused(tmp_path, monkeypatch): + jit = _jit(tmp_path, authorized=False) + program = jit.program_dir + candidate = json.loads((program / "candidate.json").read_text()) + candidate.pop("recorder") + fields = support.draft_fields(program, jit.key_dir, "t.add_note") + fields["teach_scope"]["scope"]["authorization"] = None # an unsigned scope + candidate.update(fields) + candidate = lifecycle.sign_recorder(program, candidate, key_dir=jit.key_dir) + (program / "candidate.json").write_text(json.dumps(candidate)) + _review(jit) + _stub_runs(jit, monkeypatch, _report()) + jit.canary(principal_id=REVIEWER) + assert "teach_auth_not_bound" in _refusal(jit, monkeypatch) + + +# -- a repaired copy (self-repair's carried admission) --------------------------- + + +def _repair_copy(jit, tmp_path: Path) -> Path: + copy = tmp_path / "repaired" + shutil.copytree(jit.bundle_dir, copy) + (copy / lifecycle.ADMISSION_FILE).unlink() + shutil.rmtree(copy / lifecycle.TEACH_AUTHORIZATION_DIR) + return copy + + +def test_a_repair_carries_an_authorization_that_covers_repair(tmp_path, monkeypatch): + jit = _ready(tmp_path, monkeypatch) + jit.admit_with_key(_ci(jit, monkeypatch)) + copy = _repair_copy(jit, tmp_path) + manifest = admit_repair( + jit.bundle_dir, + copy, + candidate_id="c1", + approved_by=REVIEWER, + key_dir=jit.key_dir, + ) + assert manifest["teach_authorization"]["sha256"] + assert (copy / lifecycle.TEACH_AUTHORIZATION_DIR / ta.STATEMENT_FILE).is_file() + + +def test_a_repair_needs_the_repair_purpose(tmp_path, monkeypatch): + jit = _jit(tmp_path, authorized=False) + program = jit.program_dir + candidate = json.loads((program / "candidate.json").read_text()) + candidate.pop("recorder") + candidate.update( + support.draft_fields(program, jit.key_dir, "t.add_note", purposes=("teach",)) + ) + candidate = lifecycle.sign_recorder(program, candidate, key_dir=jit.key_dir) + (program / "candidate.json").write_text(json.dumps(candidate)) + _review(jit) + _stub_runs(jit, monkeypatch, _report()) + jit.canary(principal_id=REVIEWER) + assert jit.admit_with_key(_ci(jit, monkeypatch))["outcome"] == "admitted" + copy = _repair_copy(jit, tmp_path) + with pytest.raises(AdmissionError, match="teach_auth_purpose"): + admit_repair( + jit.bundle_dir, + copy, + candidate_id="c1", + probe_only=True, + key_dir=jit.key_dir, + ) + + +def test_a_repair_after_the_authorizer_was_removed_is_refused(tmp_path, monkeypatch): + jit = _ready(tmp_path, monkeypatch) + jit.admit_with_key(_ci(jit, monkeypatch)) + support.roster_root(jit.key_dir).remove(support.authorizer(jit.key_dir)) + with pytest.raises(AdmissionError, match="teach_auth_unknown_signer"): + admit_repair( + jit.bundle_dir, + _repair_copy(jit, tmp_path), + candidate_id="c1", + approved_by=REVIEWER, + key_dir=jit.key_dir, + )