diff --git a/MODULE.md b/MODULE.md index cdf5c62..dbae4ef 100644 --- a/MODULE.md +++ b/MODULE.md @@ -5,10 +5,10 @@ druggability dossier for one protein target and writes it as a single JSON file that validates against `schemas/output.schema.json`, with a side-panel interpretability object at `output.interpretability`. -## The one command +## The command ```bash -python -m simulation run --input input.json --output output.json +python -m simulation run --mode live|replay --input input.json --output output.json ``` Copy-pasteable example against the shipped example request: @@ -16,10 +16,11 @@ Copy-pasteable example against the shipped example request: ```bash # from the station root (managed/druggability-dossier/), with the env from below active micromamba run -n druggability-simulation \ - python -m simulation run --input examples/input.json --output /tmp/dossier.json + python -m simulation run --mode replay \ + --input examples/input.json --output /tmp/dossier.json ``` -`--input` and `--output` are both required paths. `--input` must satisfy +`--mode`, `--input`, and `--output` are required. `--input` must satisfy `schemas/input.schema.json` (one required field, `uniprot_accession`). ## Exit codes @@ -28,8 +29,8 @@ micromamba run -n druggability-simulation \ | --- | --- | | `0` | success — dossier written, and BOTH the output schema and the interpretability schema validated | | `2` | invalid input — request missing/failing `schemas/input.schema.json`; **nothing is written** | -| `3` | dossier production failed — the managed agent could not be invoked (see below); **nothing is written** | -| `4` | validation failed — the dossier was written to `--output` for inspection, but it (or its interpretability object) did not validate | +| `3` | dossier production failed — a `simulation.execution-error.v1` terminal object is written with the exact reason code | +| `4` | provider/replay output was invalid — a `simulation.execution-error.v1` terminal object is written with `reasonCode: INVALID_OUTPUT` instead of publishing the invalid dossier | | `1` | usage error | Any nonzero code is a failure. Exit `0` is returned only when the dossier and its @@ -85,16 +86,26 @@ Both `simulation/requirements.txt` (exact versions) and ## How the dossier is produced (and what a live run needs) -`run_pipeline` in `simulation/pipeline.py` invokes the **Claude Managed Agent** -that is this station — there is no pure-Python re-run of the science. It drives -the documented headless route (`bun run console druggability-dossier -- --once -""`, README step 3 / `scripts/console.ts`), which runs the deployed agent, +`--mode live` invokes the **existing Claude Managed Agent** that is this station; +there is no pure-Python re-run of the science. Set `LABRADOR_RUNTIME_ROOT` to the +full LABrador checkout that already contains the managed-agent runtime and an +existing deployment. The split station drives that checkout's documented +headless route (`bun scripts/console.ts small-molecule-tractability-review -- +--once ""`), which runs the deployed agent, answers any custom-tool round-trips in-process, prints the agent's final reply (the dossier JSON) to stdout, and logs to stderr; the module then parses that JSON. A live run therefore needs `ANTHROPIC_API_KEY`, network access, `bun` on PATH, and the agent to have been deployed (`manifest.deployment.agent_id` set). When any of those is absent, `run_pipeline` raises a typed error and the command -fails loudly (exit `3`) rather than hanging or fabricating a dossier. +fails loudly (exit `3`) rather than hanging, deploying anything, or falling +back to replay. Stable terminal codes distinguish runtime, deployment, +credential, dependency, timeout, provider, and invalid-output failures. The +managed-provider session receives a 90-minute limit; the orchestrator owns the +90-minute node timeout. + +`--mode replay` uses only the bundled real-dossier cache. Cache hits are stamped +`CACHED_DOSSIER`; misses return an honest `insufficient_evidence` dossier with +no invented scientific values. The **schemas, examples, and interpretability logic are self-contained** and need none of that: they run offline against the checked-in `examples/` and fixtures, @@ -106,9 +117,8 @@ which is what `simulation/test_module.py` exercises. micromamba run -n druggability python simulation/test_module.py ``` -Offline, no paid calls: it monkeypatches `run_pipeline` with a recorded real -dossier (`examples/output.json`) to check the end-to-end contract (exit 0, output -written, stdout empty, output validates), checks that malformed input and a -raising pipeline both fail loudly with nothing written, and checks that +Offline, no paid calls: it runs replay against the bundled real dossier and +checks the end-to-end contract (exit 0, output written, stdout empty, output +validates), checks malformed input and exact live terminal errors, and checks that `build_interpretability` validates against `schemas/interpretability.schema.json` for the example dossier and both integration fixtures. diff --git a/README.md b/README.md index 3fdf4d6..4fc2489 100644 --- a/README.md +++ b/README.md @@ -61,14 +61,40 @@ Additional skills support this core pipeline, including `falsification-sweep` ## Input / output contract -The input and output are JSON. The formal JSON Schemas live under `schema/`: +The input and output are JSON. The formal JSON Schemas live under `schemas/`: -- `schema/input.schema.json` — the request contract. -- `schema/output.schema.json` — the dossier contract. +- `schemas/input.schema.json` — the request contract. +- `schemas/output.schema.json` — the dossier contract. The `input` block is echoed back verbatim on every run and is never inferred. See `CLAUDE.md` for the full field-by-field contract and operating rules. +## Run it + +Choose the mode explicitly; live mode never falls back to replay: + +```bash +# Deterministic bundled cache; no provider calls. +python -m simulation run --mode replay \ + --input examples/input.json --output /tmp/dossier.json + +# Existing Paperclip/Proto/Modal-backed managed agent. This does not deploy it. +LABRADOR_RUNTIME_ROOT=/path/to/LABrador \ + python -m simulation run --mode live \ + --input examples/input.json --output /tmp/dossier.json +``` + +A failed live run exits nonzero and writes a small +`simulation.execution-error.v1` object with `status: CANNOT_COMPLETE` and an +exact `reasonCode`. A replay cache hit remains labelled `CACHED_DOSSIER`; it is +never reported as live. + +Every successful dossier is also validated against the exact shared +interpretability schema vendored from `platform-contracts`; its source commit +and SHA-256 are recorded in `schemas/contract.lock.json`. Invalid provider +output becomes terminal `CANNOT_COMPLETE / INVALID_OUTPUT` rather than being +published as a scientific dossier. + ## Not a substitute for experiment This station reports computational and retrieved evidence about small-molecule diff --git a/manifest.json b/manifest.json index f5e0498..0a6f5b4 100644 --- a/manifest.json +++ b/manifest.json @@ -5,6 +5,16 @@ "invocation": "outcome", "session_policy": "fresh", "max_iterations": 3, + "local_cli": { + "command": "python -m simulation run --mode live|replay --input --output ", + "live_runtime_root_env": "LABRADOR_RUNTIME_ROOT", + "managed_agent_names": [ + "small-molecule-tractability-review", + "druggability-dossier" + ], + "deploys_agent": false, + "orchestrator_timeout_seconds": 5400 + }, "mcp_servers": [], "runtime_notes": [ "Custom-tool handlers run in the calling process, not the sandbox (lib/claude-managed-agent.ts consumeUntilEndTurn -> executeCustomTool). That is what lets this agent reach the paperclip binary and the fpocket/mdpocket conda stack, neither of which the sandbox can install.", diff --git a/schemas/README.md b/schemas/README.md index 41a2ae6..ccf06ce 100644 --- a/schemas/README.md +++ b/schemas/README.md @@ -1,13 +1,15 @@ # Simulation-station JSON Schemas -The machine-readable contract for the simulation (druggability-dossier) station. -Three files, all [JSON Schema draft 2020-12](https://json-schema.org/): +The machine-readable contract for the simulation (druggability-dossier) station: +three [JSON Schema draft 2020-12](https://json-schema.org/) documents plus one +source lock. | file | what it governs | | --- | --- | | `input.schema.json` | the **request** a caller sends the station | | `output.schema.json` | the **dossier** the station returns | | `interpretability.schema.json` | the **`output.interpretability`** object — the LABrador shared interpretability contract (v1.0.0) | +| `contract.lock.json` | exact `platform-contracts` source commit, path, and SHA-256 for the vendored interpretability schema | ## The interpretability contract (required) @@ -28,6 +30,11 @@ dossier — no recomputation, no fabrication; unknowns stay `null` and earn a `basis` OBSERVED|INFERRED|MODELED|SYNTHETIC; `direction` positive|negative|neutral|mixed|unknown; `grade` HIGH|MODERATE|LOW|UNSUPPORTED; `severity` INFO|WARNING|ERROR. +This repository vendors the contract byte-for-byte from +`REagent-LABrador/platform-contracts` commit +`755499b42ab65d3b01f959b11624dd4e61bdd561`. The expected SHA-256 is recorded +in `contract.lock.json` and enforced by the offline module tests. + ## What these are, and what they are not These schemas enforce **shape, types, and vocabularies** — the structural diff --git a/schemas/contract.lock.json b/schemas/contract.lock.json new file mode 100644 index 0000000..fafa92a --- /dev/null +++ b/schemas/contract.lock.json @@ -0,0 +1,11 @@ +{ + "schemaVersion": "labrador.contract-lock.v1", + "sourceRepository": "https://github.com/REagent-LABrador/platform-contracts", + "sourceCommit": "755499b42ab65d3b01f959b11624dd4e61bdd561", + "contracts": { + "interpretability.schema.json": { + "sourcePath": "schemas/interpretability.schema.json", + "sha256": "ac7b27908688851b4fc3de5e3d31642a6e9d4422b422f57161f2c9ab42c3d6bb" + } + } +} diff --git a/schemas/interpretability.schema.json b/schemas/interpretability.schema.json index cf77c6b..da6b5b3 100644 --- a/schemas/interpretability.schema.json +++ b/schemas/interpretability.schema.json @@ -1,218 +1,662 @@ { "$schema": "https://json-schema.org/draft/2020-12/schema", - "$id": "https://github.com/REagent-LABrador/simulation/schemas/interpretability.schema.json", - "title": "LABrador shared interpretability object (Tractability / Simulation module)", - "description": "The per-run object carried at output.interpretability. It answers, for the shared UI: what did the module conclude, why, on what evidence and assumptions, how each value was derived, what uncertainty and limitations remain, and what would change the conclusion. It is a WORKFLOW/decision trace, never LLM chain-of-thought. This is the common LABrador contract (schema_version 1.0.0); it SUPERSEDES this module's earlier axes/trace-shaped object, whose content now lives under `extensions`. Module-specific data (the two evidence axes kept separate, the ordered stage trace, structured identifiers, the input-hash cache key) lives in `extensions`, which the shared UI must not require.", + "$id": "https://schemas.reagent-labrador.org/interpretability/1.0.0/interpretability.schema.json", + "title": "LABrador shared interpretability contract (unified v1.0.0)", "type": "object", "required": [ - "schema_version", "headline", "metrics", "steps", "evidence", "assumptions", - "uncertainty", "limitations", "counterfactuals", "lineage", "extensions" + "schema_version", + "headline", + "metrics", + "steps", + "evidence", + "assumptions", + "uncertainty", + "limitations", + "counterfactuals", + "lineage", + "extensions" ], - "additionalProperties": true, + "additionalProperties": false, + "patternProperties": { + "^_": {} + }, "properties": { - "schema_version": {"type": "string", "const": "1.0.0"}, - + "schema_version": { + "type": "string", + "pattern": "^[0-9]+\\.[0-9]+\\.[0-9]+$" + }, "headline": { - "type": "object", - "required": ["title", "result", "plain_language", "status", "basis"], - "additionalProperties": true, - "properties": { - "title": {"type": "string", "minLength": 1}, - "result": {"type": ["string", "null"], "description": "Stable machine-readable result (this module: the verdict)."}, - "plain_language": {"type": "string", "minLength": 1}, - "status": {"enum": ["SUPPORTED", "QUALIFIED", "INCONCLUSIVE", "FAILED", "NOT_APPLICABLE"]}, - "basis": {"type": "array", "minItems": 1, "items": {"enum": ["OBSERVED", "INFERRED", "MODELED", "SYNTHETIC"]}} - } + "$ref": "#/$defs/headline" }, - "metrics": { "type": "array", - "items": {"$ref": "#/$defs/metric"} + "items": { + "$ref": "#/$defs/metric" + } }, - "steps": { "type": "array", - "items": {"$ref": "#/$defs/step"} + "items": { + "$ref": "#/$defs/step" + } }, - "evidence": { "type": "array", - "items": {"$ref": "#/$defs/evidence"} + "items": { + "$ref": "#/$defs/evidence" + } }, - "assumptions": { "type": "array", - "items": {"$ref": "#/$defs/assumption"} + "items": { + "$ref": "#/$defs/assumption" + } }, - "uncertainty": { - "type": "object", - "required": ["method", "intervals", "seed", "draws", "limitations"], - "additionalProperties": true, - "properties": { - "method": {"type": "string", "minLength": 1}, - "intervals": {"type": "array", "items": {"$ref": "#/$defs/interval"}}, - "seed": {"type": ["integer", "null"]}, - "draws": {"type": ["integer", "null"]}, - "limitations": {"type": "array", "items": {"type": "string"}} - } + "$ref": "#/$defs/uncertainty" }, - "limitations": { "type": "array", - "items": {"$ref": "#/$defs/limitation"} + "items": { + "$ref": "#/$defs/limitation" + } }, - "counterfactuals": { "type": "array", - "items": {"$ref": "#/$defs/counterfactual"} + "items": { + "$ref": "#/$defs/counterfactual" + } }, - "lineage": { "type": "array", - "items": {"$ref": "#/$defs/lineage"} + "items": { + "$ref": "#/$defs/lineage" + } }, - "extensions": { - "type": "object", - "description": "Module-specific structured data. The shared UI must not require any of it. For this module it carries the two evidence axes kept SEPARATE (never averaged), the ordered stage trace, next_experiment, the figure reference, structured PDB/ChEMBL/tool identifiers, and a cache_key hashed over the complete input.", - "additionalProperties": true + "type": "object" } }, - "$defs": { + "stable_id": { + "type": "string", + "minLength": 1, + "pattern": "^[A-Za-z0-9][A-Za-z0-9._:-]*$" + }, + "json_scalar": { + "type": [ + "number", + "string", + "boolean", + "null" + ] + }, + "json_value": { + "type": [ + "number", + "string", + "boolean", + "null", + "array", + "object" + ] + }, + "headline": { + "type": "object", + "required": [ + "title", + "result", + "plain_language", + "status", + "basis" + ], + "additionalProperties": false, + "patternProperties": { + "^_": {} + }, + "properties": { + "title": { + "type": "string", + "minLength": 1, + "maxLength": 240 + }, + "result": { + "type": "string", + "minLength": 1 + }, + "plain_language": { + "type": "string", + "minLength": 1 + }, + "status": { + "enum": [ + "SUPPORTED", + "QUALIFIED", + "INCONCLUSIVE", + "FAILED", + "NOT_APPLICABLE" + ] + }, + "basis": { + "type": "array", + "minItems": 1, + "uniqueItems": true, + "items": { + "enum": [ + "OBSERVED", + "INFERRED", + "MODELED", + "SYNTHETIC" + ] + } + } + } + }, "metric": { "type": "object", - "required": ["id", "label", "value", "unit", "display", "meaning", "direction", "evidence_ids", "assumption_ids"], - "additionalProperties": true, + "required": [ + "id", + "label", + "value", + "unit", + "display", + "meaning", + "direction", + "evidence_ids", + "assumption_ids" + ], + "additionalProperties": false, + "patternProperties": { + "^_": {} + }, "properties": { - "id": {"type": "string", "minLength": 1}, - "label": {"type": "string", "minLength": 1}, - "value": {"type": ["number", "string", "boolean", "null"]}, - "unit": {"type": ["string", "null"]}, - "display": {"type": ["string", "null"]}, - "meaning": {"type": "string", "minLength": 1}, - "direction": {"enum": ["positive", "negative", "neutral", "mixed", "unknown"]}, - "evidence_ids": {"type": "array", "items": {"type": "string"}}, - "assumption_ids": {"type": "array", "items": {"type": "string"}} + "id": { + "$ref": "#/$defs/stable_id" + }, + "label": { + "type": "string", + "minLength": 1 + }, + "value": { + "$ref": "#/$defs/json_scalar" + }, + "unit": { + "type": [ + "string", + "null" + ], + "minLength": 1 + }, + "display": { + "type": [ + "string", + "null" + ], + "minLength": 1 + }, + "meaning": { + "type": "string", + "minLength": 1 + }, + "direction": { + "enum": [ + "positive", + "negative", + "neutral", + "mixed", + "unknown" + ] + }, + "evidence_ids": { + "type": "array", + "items": { + "$ref": "#/$defs/stable_id" + } + }, + "assumption_ids": { + "type": "array", + "items": { + "$ref": "#/$defs/stable_id" + } + } + }, + "if": { + "properties": { + "value": { + "type": "number" + } + }, + "required": [ + "value" + ] }, - "comment": "Every NUMERIC metric must carry a non-empty unit.", - "if": {"properties": {"value": {"type": "number"}}}, - "then": {"properties": {"unit": {"type": "string", "minLength": 1}}, "required": ["unit"]} + "then": { + "properties": { + "unit": { + "type": "string", + "minLength": 1 + } + }, + "required": [ + "unit" + ] + } }, - "step": { "type": "object", - "required": ["id", "label", "method", "formula", "inputs", "result", "evidence_ids", "assumption_ids"], - "additionalProperties": true, + "required": [ + "id", + "label", + "method", + "formula", + "inputs", + "result", + "evidence_ids", + "assumption_ids" + ], + "additionalProperties": false, + "patternProperties": { + "^_": {} + }, "properties": { - "id": {"type": "string", "minLength": 1}, - "label": {"type": "string", "minLength": 1}, - "method": {"type": "string", "minLength": 1}, - "formula": {"type": ["string", "null"]}, + "id": { + "$ref": "#/$defs/stable_id" + }, + "label": { + "type": "string", + "minLength": 1 + }, + "method": { + "type": "string", + "minLength": 1 + }, + "formula": { + "type": [ + "string", + "null" + ] + }, "inputs": { "type": "array", "items": { "type": "object", - "required": ["path", "value", "unit"], - "additionalProperties": true, + "required": [ + "path", + "value", + "unit" + ], + "additionalProperties": false, + "patternProperties": { + "^_": {} + }, "properties": { - "path": {"type": "string"}, - "value": {"type": ["number", "string", "boolean", "null"]}, - "unit": {"type": ["string", "null"]} + "path": { + "type": "string", + "minLength": 1 + }, + "value": { + "$ref": "#/$defs/json_value" + }, + "unit": { + "type": [ + "string", + "null" + ], + "minLength": 1 + } } } }, "result": { "type": "object", - "required": ["value", "unit"], - "additionalProperties": true, + "required": [ + "value", + "unit" + ], + "additionalProperties": false, + "patternProperties": { + "^_": {} + }, "properties": { - "value": {"type": ["number", "string", "boolean", "null"]}, - "unit": {"type": ["string", "null"]} + "value": { + "$ref": "#/$defs/json_scalar" + }, + "unit": { + "type": [ + "string", + "null" + ], + "minLength": 1 + } } }, - "evidence_ids": {"type": "array", "items": {"type": "string"}}, - "assumption_ids": {"type": "array", "items": {"type": "string"}} + "evidence_ids": { + "type": "array", + "items": { + "$ref": "#/$defs/stable_id" + } + }, + "assumption_ids": { + "type": "array", + "items": { + "$ref": "#/$defs/stable_id" + } + } } }, - "evidence": { "type": "object", - "required": ["id", "claim", "source_type", "source_id", "source_url", "locator", "quote", "grade", "synthetic"], - "additionalProperties": true, + "required": [ + "id", + "claim", + "source_type", + "source_id", + "source_url", + "locator", + "quote", + "grade", + "synthetic" + ], + "additionalProperties": false, + "patternProperties": { + "^_": {} + }, "properties": { - "id": {"type": "string", "minLength": 1}, - "claim": {"type": "string", "minLength": 1}, - "source_type": {"type": "string", "minLength": 1}, - "source_id": {"type": ["string", "null"]}, - "source_url": {"type": ["string", "null"]}, - "locator": {"type": ["string", "null"]}, - "quote": {"type": ["string", "null"]}, - "grade": {"enum": ["HIGH", "MODERATE", "LOW", "UNSUPPORTED"]}, - "synthetic": {"type": "boolean"} - } - }, - + "id": { + "$ref": "#/$defs/stable_id" + }, + "claim": { + "type": "string", + "minLength": 1 + }, + "source_type": { + "type": "string", + "minLength": 1 + }, + "source_id": { + "type": [ + "string", + "null" + ], + "minLength": 1 + }, + "source_url": { + "type": [ + "string", + "null" + ], + "minLength": 1 + }, + "locator": { + "type": [ + "string", + "null" + ], + "minLength": 1 + }, + "quote": { + "type": [ + "string", + "null" + ] + }, + "grade": { + "enum": [ + "HIGH", + "MODERATE", + "LOW", + "UNSUPPORTED" + ] + }, + "synthetic": { + "type": "boolean" + } + } + }, "assumption": { "type": "object", - "required": ["id", "path", "value", "unit", "basis", "synthetic"], - "additionalProperties": true, + "required": [ + "id", + "path", + "value", + "unit", + "basis", + "synthetic" + ], + "additionalProperties": false, + "patternProperties": { + "^_": {} + }, "properties": { - "id": {"type": "string", "minLength": 1}, - "path": {"type": "string", "minLength": 1}, - "value": {"type": ["number", "string", "boolean", "null"]}, - "unit": {"type": ["string", "null"]}, - "basis": {"type": "string", "minLength": 1}, - "synthetic": {"type": "boolean"} + "id": { + "$ref": "#/$defs/stable_id" + }, + "path": { + "type": "string", + "minLength": 1 + }, + "value": { + "$ref": "#/$defs/json_value" + }, + "unit": { + "type": [ + "string", + "null" + ], + "minLength": 1 + }, + "basis": { + "type": "string", + "minLength": 1 + }, + "synthetic": { + "type": [ + "boolean", + "null" + ] + } } }, - "interval": { "type": "object", - "required": ["metric_id", "low", "central", "high", "unit", "confidence_level"], - "additionalProperties": true, + "required": [ + "metric_id", + "low", + "central", + "high", + "unit", + "confidence_level" + ], + "additionalProperties": false, + "patternProperties": { + "^_": {} + }, + "properties": { + "metric_id": { + "$ref": "#/$defs/stable_id" + }, + "low": { + "type": [ + "number", + "null" + ] + }, + "central": { + "type": [ + "number", + "null" + ] + }, + "high": { + "type": [ + "number", + "null" + ] + }, + "unit": { + "type": [ + "string", + "null" + ], + "minLength": 1 + }, + "confidence_level": { + "type": [ + "number", + "null" + ], + "minimum": 0, + "maximum": 1 + }, + "interval_type": { + "enum": [ + "percentile", + "confidence_interval", + "scenario", + "observed_range", + "heuristic_spread" + ] + } + } + }, + "uncertainty": { + "type": "object", + "required": [ + "method", + "intervals", + "seed", + "draws", + "limitations" + ], + "additionalProperties": false, + "patternProperties": { + "^_": {} + }, "properties": { - "metric_id": {"type": "string", "minLength": 1}, - "low": {"type": ["number", "null"]}, - "central": {"type": ["number", "null"]}, - "high": {"type": ["number", "null"]}, - "unit": {"type": ["string", "null"]}, - "confidence_level": {"type": ["number", "null"], "description": "Set only for a genuine confidence interval; null for a scenario or percentile range (identified in uncertainty.limitations)."} + "method": { + "type": "string", + "minLength": 1 + }, + "intervals": { + "type": "array", + "items": { + "$ref": "#/$defs/interval" + } + }, + "seed": { + "type": [ + "integer", + "string", + "null" + ] + }, + "draws": { + "type": [ + "integer", + "null" + ], + "minimum": 0 + }, + "limitations": { + "type": "array", + "minItems": 1, + "items": { + "type": "string", + "minLength": 1 + } + } } }, - "limitation": { "type": "object", - "required": ["code", "severity", "message", "field_path"], - "additionalProperties": true, + "required": [ + "code", + "severity", + "message", + "field_path" + ], + "additionalProperties": false, + "patternProperties": { + "^_": {} + }, "properties": { - "code": {"type": "string", "minLength": 1}, - "severity": {"enum": ["INFO", "WARNING", "ERROR"]}, - "message": {"type": "string", "minLength": 1}, - "field_path": {"type": ["string", "null"]} + "code": { + "type": "string", + "pattern": "^[A-Z][A-Z0-9_]*$" + }, + "severity": { + "enum": [ + "INFO", + "WARNING", + "ERROR" + ] + }, + "message": { + "type": "string", + "minLength": 1 + }, + "field_path": { + "type": [ + "string", + "null" + ], + "minLength": 1 + } } }, - "counterfactual": { "type": "object", - "required": ["change", "result", "meaning"], - "additionalProperties": true, + "required": [ + "change", + "result", + "meaning" + ], + "additionalProperties": false, + "patternProperties": { + "^_": {} + }, "properties": { - "change": {"type": ["string", "null"]}, - "result": {"type": ["string", "null"]}, - "meaning": {"type": ["string", "null"]} + "change": { + "type": "string", + "minLength": 1 + }, + "result": { + "type": "string", + "minLength": 1 + }, + "meaning": { + "type": "string", + "minLength": 1 + } } }, - "lineage": { "type": "object", - "required": ["output_path", "input_paths", "transformation"], - "additionalProperties": true, + "required": [ + "output_path", + "input_paths", + "transformation" + ], + "additionalProperties": false, + "patternProperties": { + "^_": {} + }, "properties": { - "output_path": {"type": "string", "minLength": 1}, - "input_paths": {"type": "array", "items": {"type": "string"}}, - "transformation": {"type": "string", "minLength": 1} + "output_path": { + "type": "string", + "minLength": 1 + }, + "input_paths": { + "type": "array", + "items": { + "type": "string", + "minLength": 1 + } + }, + "transformation": { + "type": "string", + "minLength": 1 + } } } } diff --git a/simulation/__main__.py b/simulation/__main__.py index b218704..8bdd03c 100644 --- a/simulation/__main__.py +++ b/simulation/__main__.py @@ -1,8 +1,8 @@ """`python -m simulation` entrypoint. -The ONE documented command: +The documented command: - python -m simulation run --input input.json --output output.json + python -m simulation run --mode live|replay --input input.json --output output.json Exit code 0 for success, nonzero for failure. All machine-readable results are written to the --output file. Human-facing logging goes to STDERR; STDOUT stays @@ -86,14 +86,19 @@ def _run(args: argparse.Namespace) -> int: # --- (b) Produce the dossier via the managed agent. Fail loudly, never fabricate. try: - dossier = run_pipeline(request) + dossier = run_pipeline(request, mode=args.mode) except PipelineError as exc: - log.error("dossier production failed: %s", exc) - log.error("nothing written to %s", output_path) + _write_execution_error(output_path, mode=args.mode, error=exc) + log.error("CANNOT_COMPLETE %s: %s", exc.code, exc) + log.error("terminal execution error written to %s", output_path) return EXIT_PIPELINE_FAILED except Exception as exc: # noqa: BLE001 - surface any unexpected pipeline error to stderr - log.error("unexpected error producing dossier: %s: %s", type(exc).__name__, exc) - log.error("nothing written to %s", output_path) + terminal_error = PipelineError( + f"{type(exc).__name__}: {exc}", code="INTERNAL_ERROR" + ) + _write_execution_error(output_path, mode=args.mode, error=terminal_error) + log.error("CANNOT_COMPLETE INTERNAL_ERROR: %s", terminal_error) + log.error("terminal execution error written to %s", output_path) return EXIT_PIPELINE_FAILED if not isinstance(dossier, dict): @@ -109,9 +114,12 @@ def _run(args: argparse.Namespace) -> int: try: dossier["interpretability"] = build_interpretability(dossier) except Exception as exc: # noqa: BLE001 - log.error("failed to build interpretability object: %s: %s", type(exc).__name__, exc) - # Still write the dossier below so it can be inspected; then fail. - _write_output(dossier, output_path) + error = PipelineError( + f"failed to build interpretability object: {type(exc).__name__}: {exc}", + code="INVALID_OUTPUT", + ) + _write_execution_error(output_path, mode=args.mode, error=error) + log.error("CANNOT_COMPLETE INVALID_OUTPUT: %s", error) return EXIT_VALIDATION_FAILED # --- (d) Validate the dossier and its interpretability object; write regardless. @@ -128,25 +136,27 @@ def _run(args: argparse.Namespace) -> int: output_errors = [] interp_errors = [] - _write_output(dossier, output_path) - - ok = True if output_errors: - ok = False log.error("dossier does not satisfy schemas/output.schema.json:") for line in output_errors: log.error("%s", line) if interp_errors: - ok = False log.error("interpretability object does not satisfy schemas/interpretability.schema.json:") for line in interp_errors: log.error("%s", line) - if not ok: - log.error("dossier written to %s for inspection, but validation FAILED", output_path) + if output_errors or interp_errors: + detail = "; ".join([*output_errors, *interp_errors]) + error = PipelineError( + f"managed/replayed dossier failed published schema validation: {detail}", + code="INVALID_OUTPUT", + ) + _write_execution_error(output_path, mode=args.mode, error=error) + log.error("CANNOT_COMPLETE INVALID_OUTPUT: terminal error written to %s", output_path) return EXIT_VALIDATION_FAILED # --- (e) Success. + _write_output(dossier, output_path) log.info("dossier written to %s; output and interpretability both valid", output_path) return EXIT_OK @@ -156,6 +166,25 @@ def _write_output(dossier: dict, output_path: Path) -> None: output_path.write_text(json.dumps(dossier, indent=2, ensure_ascii=False)) +def _write_execution_error( + output_path: Path, + *, + mode: str, + error: PipelineError, +) -> None: + """Write the small terminal envelope the orchestrator can consume on failure.""" + + terminal = { + "schemaVersion": "simulation.execution-error.v1", + "status": "CANNOT_COMPLETE", + "reasonCode": error.code, + "message": str(error), + "executionMode": mode.upper(), + } + output_path.parent.mkdir(parents=True, exist_ok=True) + output_path.write_text(json.dumps(terminal, indent=2, ensure_ascii=False)) + + def main(argv: list[str] | None = None) -> int: logging.basicConfig( level=logging.INFO, @@ -169,6 +198,12 @@ def main(argv: list[str] | None = None) -> int: ) sub = parser.add_subparsers(dest="command", required=True) run_parser = sub.add_parser("run", help="produce a dossier for one target") + run_parser.add_argument( + "--mode", + choices=("live", "replay"), + required=True, + help="live invokes the configured provider; replay uses only the bundled cache", + ) run_parser.add_argument("--input", required=True, help="path to the request JSON (input.schema.json)") run_parser.add_argument("--output", required=True, help="path to write the dossier JSON") run_parser.set_defaults(func=_run) diff --git a/simulation/cache/rhoa_P61586_gtpase_ppi.json b/simulation/cache/rhoa_P61586_gtpase_ppi.json index c69dc6a..9c596d4 100644 --- a/simulation/cache/rhoa_P61586_gtpase_ppi.json +++ b/simulation/cache/rhoa_P61586_gtpase_ppi.json @@ -600,8 +600,7 @@ ], "result": { "value": "not_tractable", - "unit": null, - "basis": "both" + "unit": null }, "evidence_ids": [ "evidence.best_potency_assay", @@ -781,4 +780,4 @@ "cache_key": "P61586|orthosteric|null" } } -} \ No newline at end of file +} diff --git a/simulation/environment.yml b/simulation/environment.yml index 1b4ee99..21b147b 100644 --- a/simulation/environment.yml +++ b/simulation/environment.yml @@ -1,16 +1,15 @@ # Locked micromamba environment for the druggability-dossier simulation module. # The module is self-contained; it depends on no other repo. # -# THE DEFAULT RUN NEEDS ONLY Python + jsonschema. `python3 -m simulation run` +# REPLAY NEEDS ONLY Python + jsonschema. `python3 -m simulation run --mode replay` # resolves the request against the bundled cache in simulation/cache/ (real # dossiers) entirely LOCALLY — no cloud, no Modal, no Paperclip, no managed # agent, no API key — so no gemmi/fpocket/GPU deps are listed and none are needed. # -# OPTIONAL — the NON-DEFAULT managed-agent path (SIMULATION_USE_AGENT=1) also -# needs the vendored TypeScript runtime at simulation/runtime/ (its handlers run -# client-side), installed separately and not covered by this file: +# LIVE uses a separately configured full LABrador checkout (its handlers run +# client-side), not covered by this file: # -# cd simulation/runtime && bun install # + bun + ANTHROPIC_API_KEY + network +# LABRADOR_RUNTIME_ROOT=/path/to/LABrador python -m simulation run --mode live ... # # This env is named distinctly so `env create -f` never clobbers the operator's # richer `druggability` env (which already carries jsonschema, so you can also diff --git a/simulation/pipeline.py b/simulation/pipeline.py index 89f56fc..52375af 100644 --- a/simulation/pipeline.py +++ b/simulation/pipeline.py @@ -1,9 +1,8 @@ -"""Produce a druggability dossier LOCALLY, dependency-light, from a bundled cache. +"""Produce a druggability dossier in one explicit execution mode. -DEFAULT RUN — no cloud deploy, no Modal, no Paperclip, no managed agent, no API -key. The default ``run_pipeline`` needs only Python's stdlib + ``jsonschema``. It -resolves the request against a BUNDLED CACHE of REAL dossiers shipped inside this -module at ``simulation/cache/``, keyed on the documented tuple +``replay`` is dependency-light and resolves the request against a BUNDLED CACHE +of REAL dossiers shipped inside this module at ``simulation/cache/``, keyed on +the documented tuple (uniprot_accession, mechanism_hypothesis, as_of_date) @@ -27,13 +26,12 @@ labrador-demo-orchestrator module-lock.json and store.py). A cache miss builds its interpretability via the deterministic ``build_interpretability``. -THE MANAGED-AGENT PATH IS RETAINED BUT NON-DEFAULT. The original cloud route -(vendored bun runner against the deployed Claude Managed Agent) is preserved as -``_run_via_agent`` and reached ONLY when the environment flag -``SIMULATION_USE_AGENT=1`` is set. The default run does not import, launch or -require any of it — no bun, no ``simulation/runtime`` install, no -``ANTHROPIC_API_KEY``, no network. The vendored runtime is left in place; the -default path simply does not depend on it. +``live`` calls the already-configured managed-agent checkout named by +``LABRADOR_RUNTIME_ROOT``. The split station never deploys or mutates that +checkout. It only runs its supported ``scripts/console.ts`` entrypoint and +fails with a stable reason code when the runtime, deployment, credential, or +provider is unavailable. There is deliberately no fallback from ``live`` to +``replay``. """ from __future__ import annotations @@ -58,28 +56,40 @@ _SCHEMA_DIR = _STATION_ROOT / "schemas" _CACHE_DIR = Path(__file__).resolve().parent / "cache" -# Env flag that opts INTO the non-default managed-agent path. Absent/anything -# other than "1" keeps the dependency-light local resolver. -_AGENT_FLAG = "SIMULATION_USE_AGENT" - -# The vendored agent runner lives beside this file, under runtime/. Only touched -# on the guarded agent path. -_RUNTIME_DIR = Path(__file__).resolve().parent / "runtime" -_RUN_ONCE = _RUNTIME_DIR / "run-once.ts" +# The live path uses the full LABrador checkout because the split repository +# contains the station contract but not the managed-agent client/runtime. +_RUNTIME_ROOT_ENV = "LABRADOR_RUNTIME_ROOT" +_LIVE_TIMEOUT_SECONDS_ENV = "SIMULATION_LIVE_TIMEOUT_SECONDS" +_DEFAULT_LIVE_TIMEOUT_SECONDS = 90 * 60 +_AGENT_NAMES = ( + "small-molecule-tractability-review", + "druggability-dossier", +) _BUN_FALLBACK = Path("/opt/homebrew/bin/bun") class PipelineError(RuntimeError): """Base class for pipeline failures.""" + code = "PIPELINE_FAILED" + + def __init__(self, message: str, *, code: str | None = None) -> None: + super().__init__(message) + if code is not None: + self.code = code + class PipelineUnavailableError(PipelineError): """The pipeline cannot run: invalid request, or (agent path) missing tooling.""" + code = "LIVE_UNAVAILABLE" + class PipelineInvocationError(PipelineError): """The agent was invoked but did not return a parseable dossier.""" + code = "PROVIDER_FAILED" + # --------------------------------------------------------------------------- # # Request validation (defensive backstop — __main__ validates first). @@ -433,29 +443,30 @@ def _resolve_local(request: dict) -> dict: # --------------------------------------------------------------------------- # # Public entrypoint. # --------------------------------------------------------------------------- # -def run_pipeline(request: dict) -> dict: - """Return a schema-valid druggability dossier for one request. - - DEFAULT (dependency-light, offline): resolve against the bundled cache; on a - miss return an honest insufficient-evidence dossier. Never fabricates a - scientific value. +def run_pipeline(request: dict, *, mode: str) -> dict: + """Return a schema-valid dossier from exactly one requested execution mode. - OPT-IN (``SIMULATION_USE_AGENT=1``): drive the vendored managed-agent runner - instead. That path needs bun, the vendored runtime and ANTHROPIC_API_KEY and - is NOT used by the default run. + ``replay`` resolves the bundled cache and never makes a provider call. + ``live`` invokes the configured managed-agent runtime and never falls back. """ _validate_request(request) - if os.environ.get(_AGENT_FLAG) == "1": - log.info("%s=1 set — using the non-default managed-agent path", _AGENT_FLAG) + if mode == "live": + log.info("live mode selected — invoking the configured managed-agent runtime") return _run_via_agent(request) - return _resolve_local(request) + if mode == "replay": + log.info("replay mode selected — resolving the bundled dossier cache") + return _resolve_local(request) + + raise PipelineUnavailableError( + f"unsupported execution mode {mode!r}; expected 'live' or 'replay'", + code="INVALID_MODE", + ) # --------------------------------------------------------------------------- # -# NON-DEFAULT managed-agent path (guarded by SIMULATION_USE_AGENT=1). -# The default run never reaches this code. +# Live managed-agent bridge. It never deploys or modifies the runtime checkout. # --------------------------------------------------------------------------- # def _resolve_bun() -> str | None: bun = shutil.which("bun") @@ -466,6 +477,79 @@ def _resolve_bun() -> str | None: return None +def _runtime_root() -> Path: + configured = os.environ.get(_RUNTIME_ROOT_ENV, "").strip() + if not configured: + raise PipelineUnavailableError( + f"{_RUNTIME_ROOT_ENV} is not set; point it at a LABrador checkout " + "that already contains the deployed tractability managed agent", + code="RUNTIME_NOT_CONFIGURED", + ) + root = Path(configured).expanduser().resolve() + if not root.is_dir(): + raise PipelineUnavailableError( + f"{_RUNTIME_ROOT_ENV} does not name a directory: {root}", + code="RUNTIME_NOT_FOUND", + ) + if not (root / "scripts" / "console.ts").is_file(): + raise PipelineUnavailableError( + f"LABrador headless runner not found at {root / 'scripts' / 'console.ts'}", + code="RUNTIME_INCOMPLETE", + ) + return root + + +def _managed_agent(root: Path) -> tuple[str, dict]: + for name in _AGENT_NAMES: + path = root / "managed" / name / "manifest.json" + if not path.is_file(): + continue + try: + manifest = json.loads(path.read_text()) + except (OSError, json.JSONDecodeError) as exc: + raise PipelineUnavailableError( + f"managed-agent manifest is unreadable at {path}: {exc}", + code="MANIFEST_INVALID", + ) from exc + if not isinstance(manifest, dict): + raise PipelineUnavailableError( + f"managed-agent manifest is not a JSON object: {path}", + code="MANIFEST_INVALID", + ) + deployment = manifest.get("deployment") + if not isinstance(deployment, dict) or not deployment.get("agent_id"): + raise PipelineUnavailableError( + f"managed agent {name!r} has no existing deployment.agent_id; " + "this runner will not deploy it", + code="DEPLOYMENT_NOT_CONFIGURED", + ) + return name, manifest + expected = ", ".join(str(root / "managed" / name / "manifest.json") for name in _AGENT_NAMES) + raise PipelineUnavailableError( + f"no tractability managed-agent manifest found; checked {expected}", + code="MANAGED_AGENT_NOT_INSTALLED", + ) + + +def _live_timeout_seconds() -> int: + raw = os.environ.get(_LIVE_TIMEOUT_SECONDS_ENV, "").strip() + if not raw: + return _DEFAULT_LIVE_TIMEOUT_SECONDS + try: + value = int(raw) + except ValueError as exc: + raise PipelineUnavailableError( + f"{_LIVE_TIMEOUT_SECONDS_ENV} must be a positive integer", + code="RUNTIME_CONFIGURATION_INVALID", + ) from exc + if value <= 0: + raise PipelineUnavailableError( + f"{_LIVE_TIMEOUT_SECONDS_ENV} must be a positive integer", + code="RUNTIME_CONFIGURATION_INVALID", + ) + return value + + def _build_task_prose(request: dict) -> str: acc = request.get("uniprot_accession") lines = [ @@ -482,7 +566,10 @@ def _build_task_prose(request: dict) -> str: def _extract_dossier_json(stdout: str) -> dict: text = stdout.strip() if not text: - raise PipelineInvocationError("agent produced no stdout to parse a dossier from") + raise PipelineInvocationError( + "agent produced no stdout to parse a dossier from", + code="INVALID_OUTPUT", + ) try: parsed = json.loads(text) if isinstance(parsed, dict): @@ -493,7 +580,8 @@ def _extract_dossier_json(stdout: str) -> dict: if obj is None: raise PipelineInvocationError( "could not find a JSON object in the agent reply; the agent must paste the " - "complete dossier JSON into its final reply (see CLAUDE.md)" + "complete dossier JSON into its final reply (see CLAUDE.md)", + code="INVALID_OUTPUT", ) return obj @@ -534,38 +622,61 @@ def _last_json_object(text: str) -> dict | None: def _run_via_agent(request: dict) -> dict: - """Invoke the deployed Claude Managed Agent via the vendored bun runner. + """Invoke the existing managed agent through its supported headless CLI.""" - Reached only when SIMULATION_USE_AGENT=1. Requires bun on PATH, the vendored - runner present, ANTHROPIC_API_KEY, network access, and a deployed agent. - Raises PipelineUnavailableError / PipelineInvocationError; never fabricates. - """ - if not _RUN_ONCE.is_file(): - raise PipelineUnavailableError( - f"vendored runner not found at {_RUN_ONCE} — the agent path is unavailable." - ) + root = _runtime_root() + agent_name, _manifest = _managed_agent(root) bun = _resolve_bun() if bun is None: raise PipelineUnavailableError( - "`bun` is not on PATH — the vendored runner cannot be launched. Install " - "bun, then `cd simulation/runtime && bun install`." + "`bun` is not on PATH, so the LABrador managed-agent runner cannot start", + code="BINARY_MISSING", + ) + if not os.environ.get("ANTHROPIC_API_KEY", "").strip(): + raise PipelineUnavailableError( + "ANTHROPIC_API_KEY is not set for live managed-agent execution", + code="CREDENTIAL_MISSING", ) task = _build_task_prose(request) - cmd = [bun, str(_RUN_ONCE), "--once", task] - log.info("invoking vendored runner (task for %s)", request.get("uniprot_accession")) + timeout_seconds = _live_timeout_seconds() + cmd = [ + bun, + "scripts/console.ts", + agent_name, + "--", + "--once", + task, + "--quiet", + "--timeout", + str(timeout_seconds), + ] + log.info( + "invoking managed agent %s for %s", + agent_name, + request.get("uniprot_accession"), + ) try: proc = subprocess.run( cmd, - cwd=str(_RUNTIME_DIR), + cwd=str(root), capture_output=True, text=True, check=False, env=os.environ.copy(), + timeout=timeout_seconds + 30, ) + except subprocess.TimeoutExpired as exc: + raise PipelineInvocationError( + f"managed-agent execution exceeded {timeout_seconds} seconds", + code="PROVIDER_TIMEOUT", + ) from exc except OSError as exc: - raise PipelineInvocationError(f"failed to launch the vendored runner: {exc}") + raise PipelineInvocationError( + f"failed to launch the managed-agent runner: {exc}", + code="RUNTIME_LAUNCH_FAILED", + ) from exc if proc.stderr: for line in proc.stderr.splitlines(): @@ -573,14 +684,52 @@ def _run_via_agent(request: dict) -> dict: if proc.returncode != 0: tail = proc.stderr[-2000:].strip() - if proc.returncode == 3 or "deployment.agent_id" in proc.stderr: + stderr_lower = proc.stderr.lower() + if "deployment.agent_id" in proc.stderr or "not deployed" in stderr_lower: + raise PipelineUnavailableError( + "the configured managed agent has no usable deployment; this runner " + "will not deploy it. Runner stderr:\n" + tail, + code="DEPLOYMENT_NOT_CONFIGURED", + ) + if "api key" in stderr_lower or "credential" in stderr_lower or "unauthorized" in stderr_lower: raise PipelineUnavailableError( - "the managed agent has not been deployed (no deployment.agent_id in " - "manifest.json). Deployment is the integrator's job — deploy it " - "(run: bun run deploy), then re-run. Runner stderr:\n" + tail + "the live provider rejected or could not find a required credential. " + "Runner stderr:\n" + tail, + code="CREDENTIAL_MISSING", + ) + if "timed out" in stderr_lower or "timeout" in stderr_lower: + raise PipelineInvocationError( + "the live provider timed out. Runner stderr:\n" + tail, + code="PROVIDER_TIMEOUT", + ) + if any( + marker in stderr_lower + for marker in ( + "not on path", + "command not found", + "no such file or directory", + "enoent", + "missing binary", + "missing dependency", + ) + ): + raise PipelineUnavailableError( + "the managed-agent runtime is missing a required dependency. " + "Runner stderr:\n" + tail, + code="DEPENDENCY_MISSING", ) raise PipelineInvocationError( - f"vendored runner exited {proc.returncode}. Runner stderr:\n{tail}" + f"managed-agent runner exited {proc.returncode}. Runner stderr:\n{tail}", + code="PROVIDER_FAILED", ) - return _extract_dossier_json(proc.stdout) + dossier = _extract_dossier_json(proc.stdout) + interp = dossier.get("interpretability") + if isinstance(interp, dict): + _stamp_extensions( + interp, + runtime_maturity="MANAGED_AGENT", + qualifiers=["LIVE"], + output_origin="live_provider", + ) + return dossier diff --git a/simulation/requirements.txt b/simulation/requirements.txt index f89ae05..a93fbbd 100644 --- a/simulation/requirements.txt +++ b/simulation/requirements.txt @@ -1,8 +1,8 @@ # Locked PYTHON dependencies for the druggability-dossier simulation module. # -# THE DEFAULT RUN NEEDS ONLY PYTHON + jsonschema. It runs LOCALLY and offline: +# REPLAY NEEDS ONLY PYTHON + jsonschema. It runs LOCALLY and offline: # -# python3 -m simulation run --input in.json --output out.json +# python3 -m simulation run --mode replay --input in.json --output out.json # # resolves the request against the bundled cache in simulation/cache/ (real # dossiers) and never touches the cloud, Modal, Paperclip, a managed agent, or an @@ -10,12 +10,11 @@ # # micromamba run -n druggability pip install -r simulation/requirements.txt # -# OPTIONAL — the managed-agent path (SIMULATION_USE_AGENT=1, NON-DEFAULT) also -# needs the vendored TypeScript runtime and its own extras, installed separately: +# LIVE uses a separately configured full LABrador checkout: # -# cd simulation/runtime && bun install # + bun on PATH + ANTHROPIC_API_KEY + network +# LABRADOR_RUNTIME_ROOT=/path/to/LABrador python -m simulation run --mode live ... # -# None of that is required for the default run. Versions below are pinned to what +# None of that is required for replay. Versions below are pinned to what # is installed in the `druggability` env (Python 3.14). jsonschema==4.26.0 # jsonschema's own dependency closure, pinned for reproducibility: diff --git a/simulation/test_module.py b/simulation/test_module.py index 1bf9192..4b7512a 100644 --- a/simulation/test_module.py +++ b/simulation/test_module.py @@ -5,10 +5,10 @@ or: micromamba run -n druggability python -m unittest simulation.test_module -The DEFAULT run is now the dependency-light LOCAL resolver over the bundled -cache in simulation/cache/ (Python + jsonschema only — no cloud, no Modal, no -Paperclip, no managed agent, no API key). So the end-to-end tests exercise the -REAL default path with NO monkeypatch: a cache hit on examples/input.json (the +Replay is the dependency-light LOCAL resolver over the bundled cache in +simulation/cache/ (Python + jsonschema only — no cloud, no Modal, no Paperclip, +no managed agent, no API key). The end-to-end tests exercise that explicit path +with NO monkeypatch: a cache hit on examples/input.json (the real IRAK4 dossier) and a cache miss on an unknown accession (an honest, schema-valid insufficient-evidence dossier). The managed-agent behaviours are still covered by monkeypatching run_pipeline, since that path is non-default and @@ -18,6 +18,7 @@ from __future__ import annotations import io +import hashlib import json import logging import sys @@ -36,6 +37,7 @@ sys.path.insert(0, str(STATION_ROOT)) import simulation.__main__ as cli # noqa: E402 +import simulation.pipeline as pipeline # noqa: E402 from simulation.interpretability import build_interpretability # noqa: E402 from simulation.pipeline import PipelineUnavailableError # noqa: E402 @@ -67,14 +69,17 @@ def _run_cli(argv: list[str]) -> tuple[int, str, str]: return code, out_buf.getvalue(), log_buf.getvalue() -class EndToEndDefaultPathCacheHit(unittest.TestCase): - """The REAL default path: no monkeypatch, offline, resolves from the cache.""" +class EndToEndReplayCacheHit(unittest.TestCase): + """The real replay path: no monkeypatch, offline, resolves from the cache.""" def test_examples_input_is_a_real_cache_hit(self): with tempfile.TemporaryDirectory() as tmp: out_path = Path(tmp) / "o.json" code, stdout, stderr = _run_cli( - ["run", "--input", str(EXAMPLES / "input.json"), "--output", str(out_path)] + [ + "run", "--mode", "replay", "--input", str(EXAMPLES / "input.json"), + "--output", str(out_path), + ] ) self.assertEqual(code, cli.EXIT_OK, msg=stderr) self.assertEqual(stdout, "", "stdout must stay empty; results go to --output") @@ -97,9 +102,32 @@ def test_examples_input_is_a_real_cache_hit(self): ext = written["interpretability"]["extensions"] self.assertEqual(ext["runtime_maturity"], "LOCAL") self.assertIn("CACHED_DOSSIER", ext["qualifiers"]) + self.assertEqual(ext["output_origin"], "cached_dossier") self.assertEqual(ext["cache_hit"]["kind"], "exact") +class SharedContractLock(unittest.TestCase): + def test_vendored_interpretability_schema_matches_platform_contract(self): + lock = json.loads((SCHEMA_DIR / "contract.lock.json").read_text()) + contract = SCHEMA_DIR / "interpretability.schema.json" + self.assertEqual( + lock["sourceCommit"], + "755499b42ab65d3b01f959b11624dd4e61bdd561", + ) + self.assertEqual( + hashlib.sha256(contract.read_bytes()).hexdigest(), + lock["contracts"][contract.name]["sha256"], + ) + + def test_every_bundled_dossier_conforms_to_shared_interpretability_schema(self): + validator = Draft202012Validator(_schema("interpretability.schema.json")) + cache_dir = STATION_ROOT / "simulation" / "cache" + for path in sorted(cache_dir.glob("*.json")): + with self.subTest(path=path.name): + dossier = json.loads(path.read_text()) + validator.validate(dossier["interpretability"]) + + class EndToEndUnknownTargetHonest(unittest.TestCase): """A target absent from the cache: valid dossier, honestly empty, no fabrication.""" @@ -115,7 +143,7 @@ def test_unknown_accession_returns_insufficient_evidence(self): })) out_path = Path(tmp) / "o.json" code, stdout, stderr = _run_cli( - ["run", "--input", str(inp), "--output", str(out_path)] + ["run", "--mode", "replay", "--input", str(inp), "--output", str(out_path)] ) self.assertEqual(code, cli.EXIT_OK, msg=stderr) self.assertEqual(stdout, "", "stdout must stay empty") @@ -151,32 +179,231 @@ def test_missing_uniprot_accession_fails_and_writes_nothing(self): cli, "run_pipeline", side_effect=AssertionError("pipeline reached on bad input") ): code, stdout, stderr = _run_cli( - ["run", "--input", str(bad_input), "--output", str(out_path)] + [ + "run", "--mode", "replay", "--input", str(bad_input), + "--output", str(out_path), + ] ) self.assertNotEqual(code, cli.EXIT_OK) self.assertFalse(out_path.exists(), "nothing must be written on invalid input") self.assertIn("input.schema.json", stderr) -class PipelineRaises(unittest.TestCase): - """The non-default agent path fails loudly and writes nothing (monkeypatched).""" +class LiveModeFailures(unittest.TestCase): + """Live mode fails terminally and never substitutes a cached dossier.""" - def test_pipeline_failure_is_loud_and_writes_nothing(self): + def test_pipeline_failure_writes_machine_readable_terminal_error(self): with tempfile.TemporaryDirectory() as tmp: out_path = Path(tmp) / "o.json" with mock.patch.object( cli, "run_pipeline", - side_effect=PipelineUnavailableError("ANTHROPIC_API_KEY is not set"), + side_effect=PipelineUnavailableError( + "ANTHROPIC_API_KEY is not set", code="CREDENTIAL_MISSING" + ), ): code, stdout, stderr = _run_cli( - ["run", "--input", str(EXAMPLES / "input.json"), "--output", str(out_path)] + [ + "run", "--mode", "live", "--input", str(EXAMPLES / "input.json"), + "--output", str(out_path), + ] ) self.assertNotEqual(code, cli.EXIT_OK) - self.assertFalse(out_path.exists(), "no dossier means nothing to write") - self.assertIn("failed", stderr.lower()) + terminal = json.loads(out_path.read_text()) + self.assertEqual(terminal["status"], "CANNOT_COMPLETE") + self.assertEqual(terminal["reasonCode"], "CREDENTIAL_MISSING") + self.assertEqual(terminal["executionMode"], "LIVE") + self.assertIn("CANNOT_COMPLETE CREDENTIAL_MISSING", stderr) self.assertIn("ANTHROPIC_API_KEY", stderr) + def test_unconfigured_live_mode_does_not_touch_replay_cache(self): + with tempfile.TemporaryDirectory() as tmp, mock.patch.dict( + "os.environ", {"LABRADOR_RUNTIME_ROOT": ""}, clear=False + ): + out_path = Path(tmp) / "o.json" + code, stdout, stderr = _run_cli( + [ + "run", "--mode", "live", "--input", str(EXAMPLES / "input.json"), + "--output", str(out_path), + ] + ) + self.assertEqual(code, cli.EXIT_PIPELINE_FAILED) + self.assertEqual(stdout, "") + terminal = json.loads(out_path.read_text()) + self.assertEqual(terminal["reasonCode"], "RUNTIME_NOT_CONFIGURED") + self.assertNotIn("CACHED_DOSSIER", out_path.read_text()) + self.assertIn("RUNTIME_NOT_CONFIGURED", stderr) + + def test_missing_existing_deployment_is_terminal_and_does_not_deploy(self): + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + (root / "scripts").mkdir() + (root / "scripts" / "console.ts").write_text("// test") + manifest_dir = root / "managed" / "small-molecule-tractability-review" + manifest_dir.mkdir(parents=True) + (manifest_dir / "manifest.json").write_text(json.dumps({})) + with mock.patch.dict( + "os.environ", {"LABRADOR_RUNTIME_ROOT": str(root)}, clear=False + ): + with self.assertRaises(PipelineUnavailableError) as raised: + pipeline.run_pipeline( + json.loads((EXAMPLES / "input.json").read_text()), + mode="live", + ) + self.assertEqual(raised.exception.code, "DEPLOYMENT_NOT_CONFIGURED") + self.assertIn("will not deploy", str(raised.exception)) + + def test_missing_provider_credential_is_exact(self): + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + (root / "scripts").mkdir() + (root / "scripts" / "console.ts").write_text("// test") + manifest_dir = root / "managed" / "small-molecule-tractability-review" + manifest_dir.mkdir(parents=True) + (manifest_dir / "manifest.json").write_text( + json.dumps({"deployment": {"agent_id": "agent-test"}}) + ) + with ( + mock.patch.dict( + "os.environ", + { + "LABRADOR_RUNTIME_ROOT": str(root), + "ANTHROPIC_API_KEY": "", + }, + clear=False, + ), + mock.patch.object(pipeline, "_resolve_bun", return_value="/usr/bin/bun"), + ): + with self.assertRaises(PipelineUnavailableError) as raised: + pipeline.run_pipeline( + json.loads((EXAMPLES / "input.json").read_text()), + mode="live", + ) + self.assertEqual(raised.exception.code, "CREDENTIAL_MISSING") + self.assertIn("ANTHROPIC_API_KEY", str(raised.exception)) + + def test_live_runner_stamps_provider_origin(self): + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + (root / "scripts").mkdir() + (root / "scripts" / "console.ts").write_text("// test") + manifest_dir = root / "managed" / "small-molecule-tractability-review" + manifest_dir.mkdir(parents=True) + (manifest_dir / "manifest.json").write_text( + json.dumps({"deployment": {"agent_id": "agent-test"}}) + ) + dossier = json.loads((EXAMPLES / "output.json").read_text()) + completed = mock.Mock(returncode=0, stdout=json.dumps(dossier), stderr="") + with ( + mock.patch.dict( + "os.environ", + { + "LABRADOR_RUNTIME_ROOT": str(root), + "ANTHROPIC_API_KEY": "test-key", + }, + clear=False, + ), + mock.patch.object(pipeline, "_resolve_bun", return_value="/usr/bin/bun"), + mock.patch.object(pipeline.subprocess, "run", return_value=completed), + ): + result = pipeline.run_pipeline( + json.loads((EXAMPLES / "input.json").read_text()), mode="live" + ) + ext = result["interpretability"]["extensions"] + self.assertEqual(ext["runtime_maturity"], "MANAGED_AGENT") + self.assertEqual(ext["output_origin"], "live_provider") + self.assertIn("LIVE", ext["qualifiers"]) + + def test_provider_timeout_has_exact_terminal_reason(self): + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + (root / "scripts").mkdir() + (root / "scripts" / "console.ts").write_text("// test") + manifest_dir = root / "managed" / "small-molecule-tractability-review" + manifest_dir.mkdir(parents=True) + (manifest_dir / "manifest.json").write_text( + json.dumps({"deployment": {"agent_id": "agent-test"}}) + ) + with ( + mock.patch.dict( + "os.environ", + { + "LABRADOR_RUNTIME_ROOT": str(root), + "ANTHROPIC_API_KEY": "test-key", + }, + clear=False, + ), + mock.patch.object(pipeline, "_resolve_bun", return_value="/usr/bin/bun"), + mock.patch.object( + pipeline.subprocess, + "run", + side_effect=pipeline.subprocess.TimeoutExpired("runner", 5400), + ), + ): + with self.assertRaises(pipeline.PipelineInvocationError) as raised: + pipeline.run_pipeline( + json.loads((EXAMPLES / "input.json").read_text()), + mode="live", + ) + self.assertEqual(raised.exception.code, "PROVIDER_TIMEOUT") + + def test_missing_runtime_dependency_has_exact_reason(self): + with tempfile.TemporaryDirectory() as tmp: + root = Path(tmp) + (root / "scripts").mkdir() + (root / "scripts" / "console.ts").write_text("// test") + manifest_dir = root / "managed" / "small-molecule-tractability-review" + manifest_dir.mkdir(parents=True) + (manifest_dir / "manifest.json").write_text( + json.dumps({"deployment": {"agent_id": "agent-test"}}) + ) + completed = mock.Mock( + returncode=1, + stdout="", + stderr="paperclip: command not found", + ) + with ( + mock.patch.dict( + "os.environ", + { + "LABRADOR_RUNTIME_ROOT": str(root), + "ANTHROPIC_API_KEY": "test-key", + }, + clear=False, + ), + mock.patch.object(pipeline, "_resolve_bun", return_value="/usr/bin/bun"), + mock.patch.object(pipeline.subprocess, "run", return_value=completed), + ): + with self.assertRaises(PipelineUnavailableError) as raised: + pipeline.run_pipeline( + json.loads((EXAMPLES / "input.json").read_text()), + mode="live", + ) + self.assertEqual(raised.exception.code, "DEPENDENCY_MISSING") + + def test_invalid_provider_output_becomes_terminal_error_not_a_dossier(self): + with tempfile.TemporaryDirectory() as tmp: + out_path = Path(tmp) / "o.json" + with mock.patch.object(cli, "run_pipeline", return_value={"unexpected": True}): + code, stdout, stderr = _run_cli( + [ + "run", + "--mode", + "live", + "--input", + str(EXAMPLES / "input.json"), + "--output", + str(out_path), + ] + ) + self.assertEqual(code, cli.EXIT_VALIDATION_FAILED) + self.assertEqual(stdout, "") + terminal = json.loads(out_path.read_text()) + self.assertEqual(terminal["status"], "CANNOT_COMPLETE") + self.assertEqual(terminal["reasonCode"], "INVALID_OUTPUT") + self.assertEqual(terminal["executionMode"], "LIVE") + self.assertIn("INVALID_OUTPUT", stderr) + class InterpretabilityValidates(unittest.TestCase): def _assert_valid(self, dossier_path: Path):