From 8496ac072d8d6d380de1a1c0699b84998313fe75 Mon Sep 17 00:00:00 2001 From: Pengfei Hu Date: Thu, 8 Oct 2026 06:20:42 -0700 Subject: [PATCH] feat(diff): name the env_vars entries that changed in an MCP row (Refs #795) An MCP server that passes variables through by name in an env_vars list changed only its list, and the row said the change was in a detail it did not show, pointing at the command's path. The grant now publishes the plain names (host-grants 0.9, extended in place, display-only) and the row names the ones gained and lost. No value is published; rows, direction, severity and check decisions do not move. --- CHANGELOG.md | 1 + STABILITY.md | 52 +++ docs/distribution-surfaces.md | 2 +- docs/host-boundary-support.md | 30 ++ docs/host-grants-inventory-schema.v0.9.json | 7 + .../core/capability_diff_rows.py | 67 +++- src/agents_shipgate/core/host_grants.py | 37 +- src/agents_shipgate/schemas/host_grants.py | 8 + tests/shard_seconds.json | 1 + tests/test_distribution_surface_parity.py | 6 + tests/test_mcp_env_var_names.py | 347 ++++++++++++++++++ 11 files changed, 546 insertions(+), 12 deletions(-) create mode 100644 tests/test_mcp_env_var_names.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 9aa5819e..90645051 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ ### Changes +- **An MCP row names the `env_vars` entries that changed.** Replaying openai/codex-security#1281, a `.mcp.json` that added `CODEX_SECURITY_PLUGIN_ROOT` to a server's `env_vars` array and changed nothing else was a `changed` row reading `no difference in the command name launch_codex_security_mcp, launch arguments, env key names or header key names; the change is in a detail this output does not show, such as the command's path or another setting`, which pointed at the command's path for a change to a variable name. An MCP server grant now publishes `env_var_names` (host-grants `0.9`, extended in place): the plain names its `env_vars` list declares, in declared order, never a `NAME=value` entry, an object, a token-shaped string or an entry the digest's redaction rewrites. The row reads `codex-security: env_vars names +CODEX_SECURITY_PLUGIN_ROOT` on `diff` (text and `--json`), `verify`, the PR comment and `check`, beside `env keys` and `header keys` (`env keys +API_BASE -DEBUG; env_vars names +X -Y`); the same names in another order read `env_vars names in a different order`, and an added or removed server's cell lists them. The label is the key the list is declared under, so no claim is made that a variable is set or read. No value is published or printed, and rotating an `env` value is still no row, because `config_sha256`'s input redacts every `env` value. The field is display-only, left out of grant equality, the inventory digests and saved baselines, so no row, direction, `expands`, severity, expansion signal or `check` decision moves; a change confined to an unpublished entry still says it is not shown. See [STABILITY.md](STABILITY.md#mcp-env-var-names-795). (#795) - **An OpenShell policy row names the declarations that changed.** Replaying openshift-kni/ai-sandbox#29, a selected policy that removed `/sandbox/.uv/python` from `filesystem_policy.read_only`, changed binary selectors and added GitHub release destinations printed `authored, OpenShell 0.1.2, 6 endpoint(s), facts → …7 endpoint(s), facts `, which names none of them. `diff` (text and `--json`), `verify`, `check` and the PR comment now print the difference: `filesystem read_only -/sandbox/.uv/python`, `destination aiplatform.googleapis.com:443 (l4) binaries +/tmp/**/node_modules/…/claude -/usr/local/bin/claude`, `removed destination oauth2.googleapis.com:443 (l4) via /usr/local/bin/claude` and `added destinations objects.githubusercontent.com:443, release-assets.githubusercontent.com:443 (l4) via …`, with changed request declarations, `include_workdir`, Landlock compatibility and process identity named the same way. Each list names its first entries and counts the rest. Only values the OpenShell inventory already publishes are named, and a value that would change under redaction prints as ``. Query and parameter constraints and endpoint options appear by field name, not value. A policy selected or removed whole lists what it declares and which sections and inspected endpoints took an upstream default, as a description and not a proof that authority grew. Display only: the published `rows[]`, direction, `expands`, severity, expansion signals and `check` decisions are unchanged, the existing unknown-direction and runtime-limit wording stays in each row's reason, and no schema changes. (#968) - **`diff` classifies a large base tree about eight times faster.** Translating each boundary glob into a regular expression was repeated for every path in the tree and was 74–86% of first-run `diff` time on two large repositories (#698's measurement). The compiled pattern is now cached per pattern; which paths match is unchanged, and a test checks the cached matcher against the previous implementation on over 10,000 random pattern and path pairs. (#698) - **`diff --application` leads with one finding per changed agent.** The text opens with a `Findings:` line and, for each agent that has a row, a finding whose first line stands alone: the tools the change added (`+`), removed (`-`) or changed (`~`), or only candidates (`?`); what they newly reach (the outbound call or library effect, its host, the credentials it sends by name, the parameters the model controls) or, for a changed tool, what it still reaches; and where reading stopped (`not read beyond memory_bank.py:531`), with the first unread hop and a count of the rest per tool. A finding with a binding or a reach it did not read is `partial`, though `comparison_status` still speaks of bindings only. A question specific to the change follows ("should `agent` reach `POST …` through `act`, sending `env WEATHER_TOKEN`, with the model controlling `city`?"; the same call newly taking a model-controlled argument; a removed tool; a changed one and what changed). Rows that are `not_established` for the same reason are one line with a count and one example, however many tools and agents share it, and limits no row states are one line per agent; effect evidence is counted once instead of `write (provisional: unknown effect)` per row. Per-row text follows under `Detail:` for up to 10 rows; above that the text says how many it did not print and `--json` carries every row, gap and reach limit. The JSON gains a top-level `summary` with the same content (`counts`, `findings`, `effects`, `causes`, `not_read`), each entry naming the rows it summarizes by index; it states nothing a row does not, and no row, status, direction, gap, scope or exit code changes. `application_comparison_schema_version` stays `0.4`, unreleased, with `summary` added. `comparison_id` digests it with the rows; `benchmark/application-q2/summarize.py` leaves it out of `answer_id`, so a hand score keeps judging the answer it read. On the 49-PR development corpus (`benchmark/application-q2/`) against `main` at `e703b41b`, `summarize.py` counts the same statuses and rows and every answer has the same `answer_id`. VidulaWickramasinghe/O.R.I.O.N#126's text goes from 11,926 lines to 24, jpka/attest#3's findings fit in 7 lines before its rows and Tiendat2703/MIS_TALENT#7's in 10; the committed goldens in `benchmark/application-q2/goldens/` hold six real answers. (#914) diff --git a/STABILITY.md b/STABILITY.md index 4c52c1ad..f2a7d184 100644 --- a/STABILITY.md +++ b/STABILITY.md @@ -693,6 +693,58 @@ unknown`. A rotation the digest's input already redacts is still no row --- + + +## Migration Note: Unreleased — an MCP row names the `env_vars` that changed (host-grants `0.9`, #795) + +**What was wrong.** An MCP server grant published the keys of its `env` map +and of its `headers` map, and nothing for an `env_vars` list. Replaying +openai/codex-security#1281, a `.mcp.json` that added +`CODEX_SECURITY_PLUGIN_ROOT` to a server's `env_vars` array and changed +nothing else was a `changed` row reading `no difference in the command name +…, launch arguments, env key names or header key names; the change is in a +detail this output does not show, such as the command's path or another +setting`, which pointed at the command's path for a change to a variable +name. + +**What changes.** See [MCP environment-variable names](docs/host-boundary-support.md#mcp-env-var-names). + +- Host-grants inventory `0.9`, unpublished before this release, is extended in + place. An `mcp_server` grant adds `env_var_names`: the entries of its + `env_vars` list that are plain environment-variable names, in declared + order, and `[]` when none are. An entry is published only when it matches + `[A-Za-z_][A-Za-z0-9_]{0,79}` and neither the redaction `config_sha256`'s + input applies nor the label redaction rewrites it, so a `NAME=value` entry, + an object, a token-shaped string and the entry after a credential word are + not published. It is display-only, like `package` and `args_sha256`: out of + grant equality, the inventory digests and saved baselines. +- A changed MCP row names the names the list gained and lost beside the + existing `env keys` and `header keys`: `codex-security: env_vars names + +CODEX_SECURITY_PLUGIN_ROOT`, `env keys +API_BASE -DEBUG; env_vars names +X + -Y`. The same names in another order read `env_vars names in a different + order`, because `config_sha256` reads the list in order. An added or removed + server's cell lists them (`docs (command name npx; env_vars names A B)`). + When no published field differs and either side lists a name, the `no + difference …` sentence names `env_vars names` among what was compared. A + change confined to an entry that is not published still says it is not + shown. +- `diff` (text and `--json`), `verify`'s host comparison, the PR comment and + `check`'s rows print the same entry. + +**Who must act.** Nobody. A reader that matched the old sentence's text, or +that validates `0.9` MCP grants against a schema generated before this change, +updates to the new one. + +**What does not change.** No value is published or printed: `env` and +`headers` values are redacted whole in `config_sha256`'s input, so rotating +one is still no row, and no per-key value digest is added. The fact is +display-only, so no row, row value, direction, `expands`, severity, +expansion signal or `check` decision moves, and an MCP edit stays `authority +direction is unknown`. No check ID, CLI flag, control state or permission is +added. `minimum_control_contract_version` stays `21`. + +--- + ## Migration Note: Unreleased — a URL in a permission rule keeps its rule (#922) diff --git a/docs/distribution-surfaces.md b/docs/distribution-surfaces.md index 28513dce..3b545031 100644 --- a/docs/distribution-surfaces.md +++ b/docs/distribution-surfaces.md @@ -74,7 +74,7 @@ and this document are checked against each other by | `human_review_request` | `docs/human-review-request.md` | `release_decision_vocabulary` | `test_surface_enumerations_match_the_engine_vocabulary` | One complete-evidence documentation-quality class only; no authority or decision ingestion. | | `human_review_decision` | `docs/human-review-decision.md` | `release_decision_vocabulary` | `test_surface_enumerations_match_the_engine_vocabulary` | Host-neutral read-only evaluator; no GitHub acquisition, persistence or operation authority. | | `github_action` | `action.yml`, `scripts/github_action_outputs.py` | `merge_verdict_vocabulary` | `test_action_input_enumerates_engine_merge_verdicts`, `test_action_output_script_shares_the_engine_merge_verdicts` | The paired `shipgate_wheel`/`shipgate_wheel_sha256` inputs install a caller-supplied local wheel instead of a published version, so that route names no channel and claims no `executable_pin`; it is refused unless both halves are given, and it installs `--no-deps`. `tests/test_action_engine_install.py` proves the refusals. Every `python` the Action starts in the workspace runs with `-P` or as a script path, so a pull request's `pip/` or `agents_shipgate/` package cannot stand in for pip or the engine; the same file executes the install and merge-verdict steps against such a checkout. The `v1.0.0` tag predates that fix; the published `v1.1.0` carries it. | -| `capability_diff` | `src/agents_shipgate/cli/diff.py`, `src/agents_shipgate/core/capability_diff_rows.py`, `src/agents_shipgate/core/host_comparison.py`, `src/agents_shipgate/report/host_comparison.py`, `src/agents_shipgate/core/unread_inputs.py`, `src/agents_shipgate/cli/verify/changed_inputs.py` | — | — | Answers no question the engine answers: it emits no verdict, no release decision and no pin. Every field is read from the drift payload the engine already produces — `risk` is the engine's severity and `expansion_signals` is the engine's word on widening — so there is no second implementation to drift. A `permission_mode` or `sandbox` row names the setting and its value as the file spells it (`enableAllProjectMcpServers: true`, `defaultMode: dontAsk`), recovered from the grant's published value and digest, and a Claude Code setting's `why` is the basis the engine's one setting table (`core/host_settings.py`) records for the value; that table also rates the grant and `check`'s violation, so a row's severity and the violation's risk give one answer (#827, `tests/test_prompt_disabling_settings.py`). `verify`/PR and `check` reuse the host comparator (#684, `tests/test_manifest_free_pr_rows.py`), and the source name each named reusable-workflow secret refers to, also non-widening, with a redacting name or target refused rather than compared, and an unreadable value neither compared nor named on this surface — only the host inventory and `audit --host` name its `job/destination`, as on `1.0.0` (#693, `tests/test_reusable_workflow_secret_mappings.py`); check retains argument redaction, except that an allow rule the engine rates as reaching a [documented arbitrary-code launcher](engineering/exec-equivalent-permissions.md) is shown in table text (`Bash(npx *)`, or a wider rule's own prefix of it, `Bash(python3 *)`) and never with a user operand, the one rating `diff`, `audit --host`, `verify` and the PR comment also carry, and its existing local-policy control (#824, `tests/test_exec_equivalent_permissions.py`). Missing comparison evidence never supplies empty comparable rows. A Claude Code permission rule row's `direction`, `expands` and `why` read the engine's one rule model (`core/claude_permission_rules.py`), the reading drift's `expansion_signals` and `check`'s violations come from, so a covered, respelled, never-consulted or carve-out rule gets one answer on every route (#918, #941, #969, #974, #938, `tests/test_claude_permission_rule_model.py`). A permission rule's cell is the text the settings reader published, never re-redacted here: a URL in it keeps its closing delimiter and trailing wildcard and withholds its path, query, fragment and userinfo, with a digest where it withholds a URL part, so two rules that publish alike stay two rows and two changes on every route (#922, `tests/test_permission_rule_url_redaction.py`). Default host mode only (`--application` is registered separately below); workflow rows compare effective writes and reusable secret recipients (#685, `tests/test_workflow_capability_diff.py`), with the privileged `pull_request_target` event stated apart from the job token scopes, a calling job's permissions printed as a ceiling whose reach the engine read from a same-repository callee's own permissions (`callee_permissions`, `callee_write_scopes` on the grant), and an inheriting call re-pinned to another reference of the same workflow a reference change rather than a new recipient, all read off the grant and `expansion_signals` the engine published (#920, #921, #924, `tests/test_actions_token_scope.py`), and each job's remote step action references, as a non-widening change (#771, `tests/test_workflow_step_action_references.py`); and each job's agent launches — a documented agent action's permission inputs, the permission flags of a `run:` that is one plain `claude -p` / `codex exec` command — and checkout refs, compared as text, as a change unless the job gains a documented widening rule, which the engine names in `expansion_signals` (`workflow_agent_widened_*`) — a rule read only from text the engine reads exactly (no shell is parsed; an argument input that is not a plain list of words is compared by a digest and read for no rule), and a gain the engine does not claim (a rule moved in from a job the launch left, one an unread step of the job rewritten as a read launch may already have met, or one the job's launch held before in an expression or an unread argument input) named in the `why` from the same engine function, never counted — with a note on a workflow row naming the untrusted-input trigger, write scopes, secrets and pull request checkout beside each agent step, read off the grant the engine published and moving no direction; an unread `run:` agent step (never compared, so never a row), an unreadable value or a setting published redacted is named only by the host inventory and `audit --host`, as for an unread secret value, an unread argument input or an unresolved launch is named there and in the `why` of a row reporting its launch, and a checkout ref holding credential-shaped text is refused as a redacting step reference is (#823, `tests/test_workflow_agent_launches.py`); every job id, step label, trigger and scope name those rows print is the label the engine published once where it built the grant, redacted, never re-derived here; `check`'s workflow evidence is derived from the raw declarations, which it still compares, and redacts job and scope names by the same rule; two distinct job ids or triggers in one workflow, or scope names in one `permissions` mapping, that publish alike are refused rather than compared, so while such a workflow exists `check` refuses on every run even when it is unchanged (#802, `tests/test_workflow_label_redaction.py`); artifact-only edits remain separate evidence. Tool-source subjects are #655. Where a partial or experimental surface is byte-identical on both sides, `diff` and `verify` compare the rest and name it in `unchanged_limits`; `check` keeps refusing, because its boundary result cannot carry a limit yet (#721). A surface the reader reaches through an in-tree link it reads through qualifies only when that link, a link with the same text at each link on the way, and the file it lands on, the same blob at the same path, are both unchanged, read from the base's Git tree entries against a commit's or, without following any link, the working tree's; any change to either is treated as before. The same proof decides which shared plugin-reference limits `check` leaves out, so behind such a link `check` compares, and publishes the rows it finds, exactly as for a limit at its own path, and a comparison `partial` only because of such a limit is `comparable` with it in `unchanged_limits` (#822, `tests/test_linked_unchanged_limits.py`). An added or changed MCP row's `why` appends a launch-source note — `launch source is mutable`, or `launch source moved from pinned … to mutable …` when both sides establish a pin — read from the `launch_source` the engine published on the grant by a bounded declaration grammar ([mcp-launch-source-notes](engineering/mcp-launch-source-notes.md)); grant equality, the inventory digests and saved baselines leave that fact out, so the note adds no row and moves no direction, `expands`, severity, expansion signal or `check` decision (#825, `tests/test_mcp_launch_source.py`). An `npx` package named with no version specifier reads instead `package spec has no exact version; launch resolution not established: npx may resolve a local project dependency or fall back to the registry/cache`, from the display-only `resolution: local_project_or_registry` the engine published beside the pin (host-grants `0.9`), as npm documents `npx` resolving such a name; no `package.json`, lockfile or cache is read, and nothing else moves (#933). A removed MCP row reads `the server is no longer declared in this source; whether it is still offered through a changed declaration this entry does not read is not established` instead of `an MCP tool surface is no longer offered to the agent` when the same comparison's #821 discovery names a changed, unread input that may declare MCP servers in the head (a plugin manifest's `mcpServers`, a manifest that does not parse, a plugin's `mcp.json`) in the same plugin scope and for the same host; the input is never read, matched or paired, keeps its coverage item, and no direction, `expands`, severity or `check` decision moves; `check` lists the changed files for it only when a server is removed and a provided diff states only the files it touches (#929, `tests/test_mcp_row_bounded_wording.py`). An MCP row's subject names the host the engine published its grant under, which for a `.mcp.json` is the host whose declaration selects it (#936): `codex` for a Codex plugin's, and no host for one the inventory published under `unknown`, whose `why` then says the host is not established; it restates the inventory's attribution, which `check`'s `affected_hosts` and `host_coverage` read too, and moves no direction, `expands`, severity, expansion signal or `check` decision (`tests/test_mcp_host_attribution.py`). A hook row's `why` states the grant's loading basis, read from its published `source`, `access` and `risk` by the engine's `hook_loading_basis`; only a hook the host loads for this project earns an expansion signal — one a settings layer declares, or one a plugin selects that the repository's project settings enable from an in-repository marketplace — so a declared-only hook, or one a plugin selects without that enablement, is a row and never an expansion, and a removal names no basis (#714). An added or changed Claude Code `PreToolUse` hook row whose basis is host configuration or a project-enabled plugin appends `inline allow auto-approves matched tool calls without a prompt (matcher …)` when a handler's published `inline_allow` is true — the engine's reading of a literal, unconditional allow on a broad matcher by a bounded declaration grammar ([inline-hook-allow-notes](engineering/inline-hook-allow-notes.md)), which grant equality, the inventory digests and saved baselines leave out — so the note adds no row and moves no direction, `expands`, severity, expansion signal or `check` decision (#826, `tests/test_inline_hook_allow.py`). A Claude Code tool-event hook row reads the `matcher_reach` the engine published on each handler (host-grants `0.9`): a hook every one of whose handlers is `no_tool_name` — its declared matcher, read by a bounded parser that never runs it, can match no built-in or MCP tool name, as with `Bash(git push*)` — earns no expansion signal, and its row is `declares a hook no tool call can trigger` and names the matcher, as does a row whose hook also has handlers that can run; the engine counts the handlers a tool call may run to decide a widening, so `diff`, `verify`, the PR comment, `check`'s rows and drift give one answer, and `check`'s decision does not move (#940, `tests/test_hook_matcher_reach.py`). A hook row also names a handler's exec-form `args` — the first argument shaped as a relative script path that no redaction rule rewrites, alone or after the arguments before it, and a digest of the rest, never other argument text — and the documented `type`, `async`, `asyncRewake`, `shell` and `once` settings it declares, before and after, read from the display-only `handlers` the engine published (host-grants `0.9`), which grant equality, the inventory digests and saved baselines leave out, so the field difference reaches the text and `review.changes[].change` and moves no row, direction, severity, expansion signal, loading basis or `check` decision, and a change confined to a field still not published, such as `if` or `statusMessage`, still says it is not shown (#972, #971, `tests/test_hook_args_and_settings.py`). A hook row also describes a changed inline command by the structure the engine published on its display-only `command.shape` (host-grants `0.9`) — the plain-token programs at its command positions, counts of commands, pipes, substitutions, control-flow keywords and quoted strings, its file redirects and a script path, never an argument, quoted string, URL or absolute path — or, when the engine published `shape_limit` instead, says the command is not described, the digest moved and the config has to be opened to read the change; the shape is a function of the text the digest holds, so every shape #987 withholds stays withheld and its rotations quiet, and it moves no row, direction, severity, expansion signal or `check` decision (#934, `tests/test_hook_command_shape.py`). `check` compares without a plugin-reference limit both sides share on an untouched source, which it cannot name and, untouched, does not route; a limit only one side carries makes its comparison incomparable. Those rows are not what routes a change: `check`, and the boundary check a manifest-backed `verify` runs, route a changed hook declaration of a plugin the project settings enable through the existing protected-surface rule, from the plugin hook reader's selection on both compared sides, and count a changed hook file such a plugin selects that the reader does not open as incomplete input; the rows beside either are unchanged (#809, `tests/test_enabled_plugin_hook_routing.py`). A partial clone that never fetched the base's objects is refused as `objects_missing`, exit `2`, never compared and never fetched; the refusal ends with the remediation sentence `verify` reports for the same reason, produced by the same function (#817, `tests/test_capability_diff_partial_clone.py`). The text of `diff`, `verify`, the PR comment and `check` reads the rows through one function, `review_changes`, and adds no row and changes no row value in any JSON projection (#795, `tests/test_host_diff_review_changes.py`): a permission rule is named with its disposition; an MCP server with the command name (never its path) or redacted URL, its package and argument digest (#819) and the env and header key names its grant already publishes, a URL printing only in the engine's sanitized scheme-and-host form and otherwise as `url not shown`, or, when none of those differ, a sentence naming what was compared and that the change is in a detail not shown, such as the command's path or another setting; a hook with each handler field that changed — its group's matcher, its command as its executable's name and digest, its timeout — before and after, a handler only one side declares, or the published handlers in a different order with a detail not shown that may also differ, and past the handler bound the same kind of sentence naming a handler past it, all read from the handlers its host-grants `0.7` grant publishes, which hold no command or argument text, and never re-derived here, and a declaration outside the documented hooks shape named as not shown rather than guessed (#819, `tests/test_hook_mcp_detail_fields.py`); those hook and MCP members display what `config_sha256` already binds, so grant equality and every inventory digest leave them out, a saved baseline holds none of them, and no row, row value, reason, digest or control answer moves; the PR comment gives the lines 1.1.0 printed their room first, the coverage block included, and prints an entry whole when the whole comment fits, otherwise cut to the widest length of at least 60 characters at which it does, or else in its shortest form (a difference cut after its name, an added or removed grant as its row), never longer than the entry 1.1.0 printed, with one line naming `verifier.json`, so no long entry hides a row, the coverage block, the change count, the review question, the reproduction or the advisory that 1.1.0 kept (#819 review, cycles 4 and 6); an allow rule the permission lattice decided another replaced (`widened` or `narrowed`), a removed and an added rule Claude Code documents as one grant in one host, source and disposition (`respelled`, #918), or the exact rule text that moved between dispositions in one host and source (`moved`), is one entry, never on the routes that redact rule arguments; and `diff` counts entries `from N rows` when one joins rows. Comparable results with entries end with one review question, naming the row count when an entry joins rows, and every result whose comparison names a base commit and a commit or working-tree head — a zero-row result and a refusal included (#812 follow-up, `tests/test_host_comparison_coverage.py`) — ends with the compared commits, the tool version and an `agents-shipgate diff --base ` reproduction, labelled `Inputs:` rather than `Compared:` where the comparison was refused, since that run compared nothing — and a refused comparison publishes no `review` object at all, so those two lines are the only place that run states its provenance, built from the `base_commit` it publishes beside the refusal; `check` and a provided diff print the question alone, and no result without a change asks a question. Every one of those facts is published beside the rows, so a machine consumer reads what a human reads (#795 slice 2, same test file): a row adds `disposition`, the `allow`/`ask`/`deny` list a permission rule is declared under and `null` for any other kind, on every route that publishes rows; and `review` in `diff --json` (capability diff `0.3`) and `host_comparison.review` in `verifier.json` (verifier `0.20`) — one object for one comparison — carry the presented changes, each naming the `row_indexes` it stands for, the `direction` the text uses (`widened`, `narrowed`, `respelled` and `moved` included, which no single row can carry), its cells, its `why` and one `expands`, plus a `summary` of `{rows, changes, widenings}` equal to `diff`'s summary line, the review question and the reproduction command. The block is refused unless its changes stand for every published row exactly once, its counters match and no joined change's two sides read alike, so the routes that redact rule arguments publish their rows alone and never a pair that reads `X → X`; `check`'s boundary result carries rows, with their dispositions, and no block. It is presentation, not a second opinion: it is the one `review_changes` projection the text prints, so the rows, their values, their count and every control answer are what they were. A comparison read back from JSON prints the changes it published, and one whose rows a caller sliced falls back to those rows. Each comparison also says what it established (#812, `tests/test_host_comparison_coverage.py`): `coverage` in `diff --json` (capability diff `0.3`) and `host_comparison.coverage` in `verifier.json` (verifier `0.20`) are the same object, printed as `What this run established` by `diff`, `verify` text and the PR comment. It is read off the grant changes, artifact changes, observed sources and blocking issues the comparator already computed: a file's rows, counting a source inside it (`#profiles.`, `#plugins.`); a file with no row and no artifact change called unchanged (`compared`, `0` rows) only when Git proves its blob identical, as the check `unchanged_limits` uses does, asked privately in one bounded batch and never published, because the artifact digest redacts `env` values and `apiKeyHelper`; a file that changed with no compared grant moving (`changed_without_grant_change`), whose artifact differs only in its digest or whose content Git shows differs while its artifact did not (never a difference a checkout line-ending conversion or a converting attribute explains, and no filter is run), never a plugin manifest or marketplace, a retargeted link or project settings while a hook's loading basis moved, worded as no compared grant changing and never as which fields changed; any other changed file with no row (`changed_without_rows`); a file Git neither proves identical nor shows differs — a provided diff, a link read, a redacted path, a working-tree file a checkout wrote with `CRLF` that Git reports unchanged — as `unchanged_not_proven`, never no change and never a change (#812 review cycle 3); the side that published a source, worded `published by` rather than `read in` for a plugin manifest or marketplace, which is published only while it declares hooks; and on a refused comparison each blocking source and its kind. Outside the bounded candidate rules below, a file no inventory observed is never an item and its absence is no claim, which the block states where it is read — one line under the heading and `read_sources_only` in the JSON — so a true list cannot be taken for the account of the change (#812 follow-up); a source already in `unchanged_limits` is not repeated; the list is capped at ten with `omitted_items`, ordered so what no row shows precedes a file's rows and, among blocking limits, by kind (`unreadable`, `parse_failed`, `unresolved_precedence`, then `unsupported`, `dynamic_source_excluded`, `remote_source_excluded`) — order, not severity, and a ranking of kinds rather than of items, since `unsupported` carries both a file this entry merely does not accept and one whose own text would not parse, so an item behind the count may still be one to repair; total down to every field an item is keyed by, the source name and then its side, limit and status — and counted in text as items not listed, ranked below those listed, and the PR comment lists only what fits in the room its entries, review question, reproduction, advisory, next action and evidence leave, at most 2000 characters, so the block never pushes out a line the comment prints without it (a row list that fills the comment by itself still truncates it, as on `1.0.0`); an instruction file's line carries no redacted-values note; sources are the inventory's redacted paths; `null` means not recorded, which is how a `0.19` verifier reads. It moves no row, reason, digest, baseline, control state or next action, and `check`'s boundary result and text carry none, so neither `check` nor a provided diff asks Git anything for it. The same list names the changed inputs this entry does not read (#821, `tests/test_unread_changed_inputs.py`): capability diff `0.4` and verifier `0.21` add a `changed_not_read` item, with the `candidate` rule that named it, for each path in the comparison's own changed-file set — the committed change, or the working tree's tracked and untracked changes — that a bounded, documented rule set recognises as plausibly agent configuration (`mcp.json` in a plugin directory, a plugin manifest's `mcpServers`, a Codex, Cursor or Copilot manifest's `hooks` and the hook files it names, a manifest or marketplace that does not parse, `.cursor/hooks.json`, host settings below the repository root, a marketplace entry's external `source`) and that no inventory published; a member is named whatever read its file, because no reader reads it. It is named from the path and, for a manifest or marketplace member, its text: nothing is fetched, run or read as a grant, so it is never a row, a widening, a `check` violation or a loading claim, and an external source is described redacted and never fetched. It ranks right after the blocking limits, inside the same cap; `read_sources_only` is `false` while one is named, and the first line says so instead; `unread_candidates` and `unread_candidates_not_examined` say whether the change set was examined and how many candidates were not — past the bound of 32, or because a file the rule needed was not read or did not parse, one count the text names both causes of. A manifest-free `verify` whose only host-relevant change is such an input, or a changed candidate it counts as not examined, publishes the comparison instead of the setup route, and `verify --preview` then names `audit --host` instead of `init --write`, in an agent-related workspace too; a `0.20` verifier reads with the search not recorded. A comparison refused only by plugin-reference limits, each bounded by its plugin directory, that no compared source depends on, is `partial` instead (#808, `tests/test_partial_host_comparison.py`); any other blocking limit it carries must be one both sides share on an unchanged source, named in `unchanged_limits` as on a comparable result. Capability diff `0.4` and verifier `0.21` publish `comparison_status: partial` with the refusal's `incomparable_reasons`, the rows, review and unchanged limits established outside those directories, and each directory (the outermost, where one holds another) as the reserved `coverage.items[].scope` on the `blocking_limit` items it bounds, and never call a changed project settings file without a row `changed_without_grant_change`, since the hooks whose loading basis it decides are not all compared; `diff`, `verify` text and the PR comment lead with `Partial comparison against …` or `Host capability comparison partial: …` and `Not compared: , …` before any entry, and a partial result with no entry is never printed as no change. Independence is read off the reader's reference graph, never off directory names: any other limit that is not unchanged, a reference leaving its plugin, a plugin at the root or holding project settings, a marketplace elsewhere declaring inline hooks for it, or a directory that does not publish as itself refuses as before. It answers no engine question and moves no control: a partial comparison is not comparable, `verify`'s control and route are the refusal's, the control envelope projects it as `incomparable` with no rows, and `check`, whose boundary result cannot name a directory, refuses its comparison and decides exactly as before. A `0.20` verifier claiming a partial comparison or a scope is refused. A plain instruction document (`AGENTS.md`, `AGENTS.override.md`, `CLAUDE.md`) both sides read directly, refused only because it is past the 256 KiB instruction bound, is withheld the same way and named as its own `scope` with `limit: unsupported`, since that profile publishes no grant and so no compared row depends on its text; a skill, command, Cursor rule, unread role, NUL byte, one-sided or linked document, or one proven unchanged (named in `unchanged_limits`) is treated as before, and a blocking limit both sides carry ends its `detail` and its line with whether Git shows its file changed in this change, identical, or neither (#973, `tests/test_oversized_instruction_document_partial.py`). A selected hook's script bytes are a comparison input of its hook grant (#702, `tests/test_hook_script_capture.py`, `tests/test_hook_script_comparison_limits.py`): a script-only edit is one `changed` row on the declaring hook, never an expansion, whose `why` names the script and loading basis and whose change carries the digests, read from the `script_inputs` the engine published; the script's own coverage line is `changed_without_rows`, which the text words as the row of the hook that runs it naming it, and a script only one side's hooks select is `compared` on that side, its selection change being the hook's row. A script the reader could not read costs that script only: a shared limit Git proves unchanged, or a script Git proves is on neither side, is an `unchanged_limits` entry of kind `unreadable`, and any other makes the comparison `partial` with a `blocking_limit` item naming the script as both `source` and `scope`, and `Not compared: the bytes of …` in the text; a working-tree difference a line-ending conversion can explain is `unchanged_not_proven`, not a row. A selected hook whose script is not resolved is a `script_not_resolved` item naming each handler and its reason, while the change could touch it, never a row, widening or limit. `check` and the boundary check of a manifest-backed `verify` route a changed selected script as a protected change of the hosts selecting it, from both compared sides' declarations, reading the base's only when the change touches a declaration (`tests/test_hook_script_routing.py`); `check` and a provided diff, which name no limit, leave out a selected script the change does not touch, from their comparison and input coverage alike, and a provided diff that touches both a script and its declaring file compares the script's bytes as the diff states them (`tests/test_hook_script_comparison_limits.py`). | +| `capability_diff` | `src/agents_shipgate/cli/diff.py`, `src/agents_shipgate/core/capability_diff_rows.py`, `src/agents_shipgate/core/host_comparison.py`, `src/agents_shipgate/report/host_comparison.py`, `src/agents_shipgate/core/unread_inputs.py`, `src/agents_shipgate/cli/verify/changed_inputs.py` | — | — | Answers no question the engine answers: it emits no verdict, no release decision and no pin. Every field is read from the drift payload the engine already produces — `risk` is the engine's severity and `expansion_signals` is the engine's word on widening — so there is no second implementation to drift. A `permission_mode` or `sandbox` row names the setting and its value as the file spells it (`enableAllProjectMcpServers: true`, `defaultMode: dontAsk`), recovered from the grant's published value and digest, and a Claude Code setting's `why` is the basis the engine's one setting table (`core/host_settings.py`) records for the value; that table also rates the grant and `check`'s violation, so a row's severity and the violation's risk give one answer (#827, `tests/test_prompt_disabling_settings.py`). `verify`/PR and `check` reuse the host comparator (#684, `tests/test_manifest_free_pr_rows.py`), and the source name each named reusable-workflow secret refers to, also non-widening, with a redacting name or target refused rather than compared, and an unreadable value neither compared nor named on this surface — only the host inventory and `audit --host` name its `job/destination`, as on `1.0.0` (#693, `tests/test_reusable_workflow_secret_mappings.py`); check retains argument redaction, except that an allow rule the engine rates as reaching a [documented arbitrary-code launcher](engineering/exec-equivalent-permissions.md) is shown in table text (`Bash(npx *)`, or a wider rule's own prefix of it, `Bash(python3 *)`) and never with a user operand, the one rating `diff`, `audit --host`, `verify` and the PR comment also carry, and its existing local-policy control (#824, `tests/test_exec_equivalent_permissions.py`). Missing comparison evidence never supplies empty comparable rows. A Claude Code permission rule row's `direction`, `expands` and `why` read the engine's one rule model (`core/claude_permission_rules.py`), the reading drift's `expansion_signals` and `check`'s violations come from, so a covered, respelled, never-consulted or carve-out rule gets one answer on every route (#918, #941, #969, #974, #938, `tests/test_claude_permission_rule_model.py`). A permission rule's cell is the text the settings reader published, never re-redacted here: a URL in it keeps its closing delimiter and trailing wildcard and withholds its path, query, fragment and userinfo, with a digest where it withholds a URL part, so two rules that publish alike stay two rows and two changes on every route (#922, `tests/test_permission_rule_url_redaction.py`). Default host mode only (`--application` is registered separately below); workflow rows compare effective writes and reusable secret recipients (#685, `tests/test_workflow_capability_diff.py`), with the privileged `pull_request_target` event stated apart from the job token scopes, a calling job's permissions printed as a ceiling whose reach the engine read from a same-repository callee's own permissions (`callee_permissions`, `callee_write_scopes` on the grant), and an inheriting call re-pinned to another reference of the same workflow a reference change rather than a new recipient, all read off the grant and `expansion_signals` the engine published (#920, #921, #924, `tests/test_actions_token_scope.py`), and each job's remote step action references, as a non-widening change (#771, `tests/test_workflow_step_action_references.py`); and each job's agent launches — a documented agent action's permission inputs, the permission flags of a `run:` that is one plain `claude -p` / `codex exec` command — and checkout refs, compared as text, as a change unless the job gains a documented widening rule, which the engine names in `expansion_signals` (`workflow_agent_widened_*`) — a rule read only from text the engine reads exactly (no shell is parsed; an argument input that is not a plain list of words is compared by a digest and read for no rule), and a gain the engine does not claim (a rule moved in from a job the launch left, one an unread step of the job rewritten as a read launch may already have met, or one the job's launch held before in an expression or an unread argument input) named in the `why` from the same engine function, never counted — with a note on a workflow row naming the untrusted-input trigger, write scopes, secrets and pull request checkout beside each agent step, read off the grant the engine published and moving no direction; an unread `run:` agent step (never compared, so never a row), an unreadable value or a setting published redacted is named only by the host inventory and `audit --host`, as for an unread secret value, an unread argument input or an unresolved launch is named there and in the `why` of a row reporting its launch, and a checkout ref holding credential-shaped text is refused as a redacting step reference is (#823, `tests/test_workflow_agent_launches.py`); every job id, step label, trigger and scope name those rows print is the label the engine published once where it built the grant, redacted, never re-derived here; `check`'s workflow evidence is derived from the raw declarations, which it still compares, and redacts job and scope names by the same rule; two distinct job ids or triggers in one workflow, or scope names in one `permissions` mapping, that publish alike are refused rather than compared, so while such a workflow exists `check` refuses on every run even when it is unchanged (#802, `tests/test_workflow_label_redaction.py`); artifact-only edits remain separate evidence. Tool-source subjects are #655. Where a partial or experimental surface is byte-identical on both sides, `diff` and `verify` compare the rest and name it in `unchanged_limits`; `check` keeps refusing, because its boundary result cannot carry a limit yet (#721). A surface the reader reaches through an in-tree link it reads through qualifies only when that link, a link with the same text at each link on the way, and the file it lands on, the same blob at the same path, are both unchanged, read from the base's Git tree entries against a commit's or, without following any link, the working tree's; any change to either is treated as before. The same proof decides which shared plugin-reference limits `check` leaves out, so behind such a link `check` compares, and publishes the rows it finds, exactly as for a limit at its own path, and a comparison `partial` only because of such a limit is `comparable` with it in `unchanged_limits` (#822, `tests/test_linked_unchanged_limits.py`). An added or changed MCP row's `why` appends a launch-source note — `launch source is mutable`, or `launch source moved from pinned … to mutable …` when both sides establish a pin — read from the `launch_source` the engine published on the grant by a bounded declaration grammar ([mcp-launch-source-notes](engineering/mcp-launch-source-notes.md)); grant equality, the inventory digests and saved baselines leave that fact out, so the note adds no row and moves no direction, `expands`, severity, expansion signal or `check` decision (#825, `tests/test_mcp_launch_source.py`). An `npx` package named with no version specifier reads instead `package spec has no exact version; launch resolution not established: npx may resolve a local project dependency or fall back to the registry/cache`, from the display-only `resolution: local_project_or_registry` the engine published beside the pin (host-grants `0.9`), as npm documents `npx` resolving such a name; no `package.json`, lockfile or cache is read, and nothing else moves (#933). A changed MCP row names the plain environment-variable names its `env_vars` list gained and lost (`env_vars names +CODEX_SECURITY_PLUGIN_ROOT`, or `in a different order`) from the display-only `env_var_names` the engine published on the grant (host-grants `0.9`), never a value; grant equality, the inventory digests and saved baselines leave it out, so it adds no row and moves no direction, `expands`, severity, expansion signal or `check` decision (#795, `tests/test_mcp_env_var_names.py`). A removed MCP row reads `the server is no longer declared in this source; whether it is still offered through a changed declaration this entry does not read is not established` instead of `an MCP tool surface is no longer offered to the agent` when the same comparison's #821 discovery names a changed, unread input that may declare MCP servers in the head (a plugin manifest's `mcpServers`, a manifest that does not parse, a plugin's `mcp.json`) in the same plugin scope and for the same host; the input is never read, matched or paired, keeps its coverage item, and no direction, `expands`, severity or `check` decision moves; `check` lists the changed files for it only when a server is removed and a provided diff states only the files it touches (#929, `tests/test_mcp_row_bounded_wording.py`). An MCP row's subject names the host the engine published its grant under, which for a `.mcp.json` is the host whose declaration selects it (#936): `codex` for a Codex plugin's, and no host for one the inventory published under `unknown`, whose `why` then says the host is not established; it restates the inventory's attribution, which `check`'s `affected_hosts` and `host_coverage` read too, and moves no direction, `expands`, severity, expansion signal or `check` decision (`tests/test_mcp_host_attribution.py`). A hook row's `why` states the grant's loading basis, read from its published `source`, `access` and `risk` by the engine's `hook_loading_basis`; only a hook the host loads for this project earns an expansion signal — one a settings layer declares, or one a plugin selects that the repository's project settings enable from an in-repository marketplace — so a declared-only hook, or one a plugin selects without that enablement, is a row and never an expansion, and a removal names no basis (#714). An added or changed Claude Code `PreToolUse` hook row whose basis is host configuration or a project-enabled plugin appends `inline allow auto-approves matched tool calls without a prompt (matcher …)` when a handler's published `inline_allow` is true — the engine's reading of a literal, unconditional allow on a broad matcher by a bounded declaration grammar ([inline-hook-allow-notes](engineering/inline-hook-allow-notes.md)), which grant equality, the inventory digests and saved baselines leave out — so the note adds no row and moves no direction, `expands`, severity, expansion signal or `check` decision (#826, `tests/test_inline_hook_allow.py`). A Claude Code tool-event hook row reads the `matcher_reach` the engine published on each handler (host-grants `0.9`): a hook every one of whose handlers is `no_tool_name` — its declared matcher, read by a bounded parser that never runs it, can match no built-in or MCP tool name, as with `Bash(git push*)` — earns no expansion signal, and its row is `declares a hook no tool call can trigger` and names the matcher, as does a row whose hook also has handlers that can run; the engine counts the handlers a tool call may run to decide a widening, so `diff`, `verify`, the PR comment, `check`'s rows and drift give one answer, and `check`'s decision does not move (#940, `tests/test_hook_matcher_reach.py`). A hook row also names a handler's exec-form `args` — the first argument shaped as a relative script path that no redaction rule rewrites, alone or after the arguments before it, and a digest of the rest, never other argument text — and the documented `type`, `async`, `asyncRewake`, `shell` and `once` settings it declares, before and after, read from the display-only `handlers` the engine published (host-grants `0.9`), which grant equality, the inventory digests and saved baselines leave out, so the field difference reaches the text and `review.changes[].change` and moves no row, direction, severity, expansion signal, loading basis or `check` decision, and a change confined to a field still not published, such as `if` or `statusMessage`, still says it is not shown (#972, #971, `tests/test_hook_args_and_settings.py`). A hook row also describes a changed inline command by the structure the engine published on its display-only `command.shape` (host-grants `0.9`) — the plain-token programs at its command positions, counts of commands, pipes, substitutions, control-flow keywords and quoted strings, its file redirects and a script path, never an argument, quoted string, URL or absolute path — or, when the engine published `shape_limit` instead, says the command is not described, the digest moved and the config has to be opened to read the change; the shape is a function of the text the digest holds, so every shape #987 withholds stays withheld and its rotations quiet, and it moves no row, direction, severity, expansion signal or `check` decision (#934, `tests/test_hook_command_shape.py`). `check` compares without a plugin-reference limit both sides share on an untouched source, which it cannot name and, untouched, does not route; a limit only one side carries makes its comparison incomparable. Those rows are not what routes a change: `check`, and the boundary check a manifest-backed `verify` runs, route a changed hook declaration of a plugin the project settings enable through the existing protected-surface rule, from the plugin hook reader's selection on both compared sides, and count a changed hook file such a plugin selects that the reader does not open as incomplete input; the rows beside either are unchanged (#809, `tests/test_enabled_plugin_hook_routing.py`). A partial clone that never fetched the base's objects is refused as `objects_missing`, exit `2`, never compared and never fetched; the refusal ends with the remediation sentence `verify` reports for the same reason, produced by the same function (#817, `tests/test_capability_diff_partial_clone.py`). The text of `diff`, `verify`, the PR comment and `check` reads the rows through one function, `review_changes`, and adds no row and changes no row value in any JSON projection (#795, `tests/test_host_diff_review_changes.py`): a permission rule is named with its disposition; an MCP server with the command name (never its path) or redacted URL, its package and argument digest (#819) and the env and header key names its grant already publishes, a URL printing only in the engine's sanitized scheme-and-host form and otherwise as `url not shown`, or, when none of those differ, a sentence naming what was compared and that the change is in a detail not shown, such as the command's path or another setting; a hook with each handler field that changed — its group's matcher, its command as its executable's name and digest, its timeout — before and after, a handler only one side declares, or the published handlers in a different order with a detail not shown that may also differ, and past the handler bound the same kind of sentence naming a handler past it, all read from the handlers its host-grants `0.7` grant publishes, which hold no command or argument text, and never re-derived here, and a declaration outside the documented hooks shape named as not shown rather than guessed (#819, `tests/test_hook_mcp_detail_fields.py`); those hook and MCP members display what `config_sha256` already binds, so grant equality and every inventory digest leave them out, a saved baseline holds none of them, and no row, row value, reason, digest or control answer moves; the PR comment gives the lines 1.1.0 printed their room first, the coverage block included, and prints an entry whole when the whole comment fits, otherwise cut to the widest length of at least 60 characters at which it does, or else in its shortest form (a difference cut after its name, an added or removed grant as its row), never longer than the entry 1.1.0 printed, with one line naming `verifier.json`, so no long entry hides a row, the coverage block, the change count, the review question, the reproduction or the advisory that 1.1.0 kept (#819 review, cycles 4 and 6); an allow rule the permission lattice decided another replaced (`widened` or `narrowed`), a removed and an added rule Claude Code documents as one grant in one host, source and disposition (`respelled`, #918), or the exact rule text that moved between dispositions in one host and source (`moved`), is one entry, never on the routes that redact rule arguments; and `diff` counts entries `from N rows` when one joins rows. Comparable results with entries end with one review question, naming the row count when an entry joins rows, and every result whose comparison names a base commit and a commit or working-tree head — a zero-row result and a refusal included (#812 follow-up, `tests/test_host_comparison_coverage.py`) — ends with the compared commits, the tool version and an `agents-shipgate diff --base ` reproduction, labelled `Inputs:` rather than `Compared:` where the comparison was refused, since that run compared nothing — and a refused comparison publishes no `review` object at all, so those two lines are the only place that run states its provenance, built from the `base_commit` it publishes beside the refusal; `check` and a provided diff print the question alone, and no result without a change asks a question. Every one of those facts is published beside the rows, so a machine consumer reads what a human reads (#795 slice 2, same test file): a row adds `disposition`, the `allow`/`ask`/`deny` list a permission rule is declared under and `null` for any other kind, on every route that publishes rows; and `review` in `diff --json` (capability diff `0.3`) and `host_comparison.review` in `verifier.json` (verifier `0.20`) — one object for one comparison — carry the presented changes, each naming the `row_indexes` it stands for, the `direction` the text uses (`widened`, `narrowed`, `respelled` and `moved` included, which no single row can carry), its cells, its `why` and one `expands`, plus a `summary` of `{rows, changes, widenings}` equal to `diff`'s summary line, the review question and the reproduction command. The block is refused unless its changes stand for every published row exactly once, its counters match and no joined change's two sides read alike, so the routes that redact rule arguments publish their rows alone and never a pair that reads `X → X`; `check`'s boundary result carries rows, with their dispositions, and no block. It is presentation, not a second opinion: it is the one `review_changes` projection the text prints, so the rows, their values, their count and every control answer are what they were. A comparison read back from JSON prints the changes it published, and one whose rows a caller sliced falls back to those rows. Each comparison also says what it established (#812, `tests/test_host_comparison_coverage.py`): `coverage` in `diff --json` (capability diff `0.3`) and `host_comparison.coverage` in `verifier.json` (verifier `0.20`) are the same object, printed as `What this run established` by `diff`, `verify` text and the PR comment. It is read off the grant changes, artifact changes, observed sources and blocking issues the comparator already computed: a file's rows, counting a source inside it (`#profiles.`, `#plugins.`); a file with no row and no artifact change called unchanged (`compared`, `0` rows) only when Git proves its blob identical, as the check `unchanged_limits` uses does, asked privately in one bounded batch and never published, because the artifact digest redacts `env` values and `apiKeyHelper`; a file that changed with no compared grant moving (`changed_without_grant_change`), whose artifact differs only in its digest or whose content Git shows differs while its artifact did not (never a difference a checkout line-ending conversion or a converting attribute explains, and no filter is run), never a plugin manifest or marketplace, a retargeted link or project settings while a hook's loading basis moved, worded as no compared grant changing and never as which fields changed; any other changed file with no row (`changed_without_rows`); a file Git neither proves identical nor shows differs — a provided diff, a link read, a redacted path, a working-tree file a checkout wrote with `CRLF` that Git reports unchanged — as `unchanged_not_proven`, never no change and never a change (#812 review cycle 3); the side that published a source, worded `published by` rather than `read in` for a plugin manifest or marketplace, which is published only while it declares hooks; and on a refused comparison each blocking source and its kind. Outside the bounded candidate rules below, a file no inventory observed is never an item and its absence is no claim, which the block states where it is read — one line under the heading and `read_sources_only` in the JSON — so a true list cannot be taken for the account of the change (#812 follow-up); a source already in `unchanged_limits` is not repeated; the list is capped at ten with `omitted_items`, ordered so what no row shows precedes a file's rows and, among blocking limits, by kind (`unreadable`, `parse_failed`, `unresolved_precedence`, then `unsupported`, `dynamic_source_excluded`, `remote_source_excluded`) — order, not severity, and a ranking of kinds rather than of items, since `unsupported` carries both a file this entry merely does not accept and one whose own text would not parse, so an item behind the count may still be one to repair; total down to every field an item is keyed by, the source name and then its side, limit and status — and counted in text as items not listed, ranked below those listed, and the PR comment lists only what fits in the room its entries, review question, reproduction, advisory, next action and evidence leave, at most 2000 characters, so the block never pushes out a line the comment prints without it (a row list that fills the comment by itself still truncates it, as on `1.0.0`); an instruction file's line carries no redacted-values note; sources are the inventory's redacted paths; `null` means not recorded, which is how a `0.19` verifier reads. It moves no row, reason, digest, baseline, control state or next action, and `check`'s boundary result and text carry none, so neither `check` nor a provided diff asks Git anything for it. The same list names the changed inputs this entry does not read (#821, `tests/test_unread_changed_inputs.py`): capability diff `0.4` and verifier `0.21` add a `changed_not_read` item, with the `candidate` rule that named it, for each path in the comparison's own changed-file set — the committed change, or the working tree's tracked and untracked changes — that a bounded, documented rule set recognises as plausibly agent configuration (`mcp.json` in a plugin directory, a plugin manifest's `mcpServers`, a Codex, Cursor or Copilot manifest's `hooks` and the hook files it names, a manifest or marketplace that does not parse, `.cursor/hooks.json`, host settings below the repository root, a marketplace entry's external `source`) and that no inventory published; a member is named whatever read its file, because no reader reads it. It is named from the path and, for a manifest or marketplace member, its text: nothing is fetched, run or read as a grant, so it is never a row, a widening, a `check` violation or a loading claim, and an external source is described redacted and never fetched. It ranks right after the blocking limits, inside the same cap; `read_sources_only` is `false` while one is named, and the first line says so instead; `unread_candidates` and `unread_candidates_not_examined` say whether the change set was examined and how many candidates were not — past the bound of 32, or because a file the rule needed was not read or did not parse, one count the text names both causes of. A manifest-free `verify` whose only host-relevant change is such an input, or a changed candidate it counts as not examined, publishes the comparison instead of the setup route, and `verify --preview` then names `audit --host` instead of `init --write`, in an agent-related workspace too; a `0.20` verifier reads with the search not recorded. A comparison refused only by plugin-reference limits, each bounded by its plugin directory, that no compared source depends on, is `partial` instead (#808, `tests/test_partial_host_comparison.py`); any other blocking limit it carries must be one both sides share on an unchanged source, named in `unchanged_limits` as on a comparable result. Capability diff `0.4` and verifier `0.21` publish `comparison_status: partial` with the refusal's `incomparable_reasons`, the rows, review and unchanged limits established outside those directories, and each directory (the outermost, where one holds another) as the reserved `coverage.items[].scope` on the `blocking_limit` items it bounds, and never call a changed project settings file without a row `changed_without_grant_change`, since the hooks whose loading basis it decides are not all compared; `diff`, `verify` text and the PR comment lead with `Partial comparison against …` or `Host capability comparison partial: …` and `Not compared: , …` before any entry, and a partial result with no entry is never printed as no change. Independence is read off the reader's reference graph, never off directory names: any other limit that is not unchanged, a reference leaving its plugin, a plugin at the root or holding project settings, a marketplace elsewhere declaring inline hooks for it, or a directory that does not publish as itself refuses as before. It answers no engine question and moves no control: a partial comparison is not comparable, `verify`'s control and route are the refusal's, the control envelope projects it as `incomparable` with no rows, and `check`, whose boundary result cannot name a directory, refuses its comparison and decides exactly as before. A `0.20` verifier claiming a partial comparison or a scope is refused. A plain instruction document (`AGENTS.md`, `AGENTS.override.md`, `CLAUDE.md`) both sides read directly, refused only because it is past the 256 KiB instruction bound, is withheld the same way and named as its own `scope` with `limit: unsupported`, since that profile publishes no grant and so no compared row depends on its text; a skill, command, Cursor rule, unread role, NUL byte, one-sided or linked document, or one proven unchanged (named in `unchanged_limits`) is treated as before, and a blocking limit both sides carry ends its `detail` and its line with whether Git shows its file changed in this change, identical, or neither (#973, `tests/test_oversized_instruction_document_partial.py`). A selected hook's script bytes are a comparison input of its hook grant (#702, `tests/test_hook_script_capture.py`, `tests/test_hook_script_comparison_limits.py`): a script-only edit is one `changed` row on the declaring hook, never an expansion, whose `why` names the script and loading basis and whose change carries the digests, read from the `script_inputs` the engine published; the script's own coverage line is `changed_without_rows`, which the text words as the row of the hook that runs it naming it, and a script only one side's hooks select is `compared` on that side, its selection change being the hook's row. A script the reader could not read costs that script only: a shared limit Git proves unchanged, or a script Git proves is on neither side, is an `unchanged_limits` entry of kind `unreadable`, and any other makes the comparison `partial` with a `blocking_limit` item naming the script as both `source` and `scope`, and `Not compared: the bytes of …` in the text; a working-tree difference a line-ending conversion can explain is `unchanged_not_proven`, not a row. A selected hook whose script is not resolved is a `script_not_resolved` item naming each handler and its reason, while the change could touch it, never a row, widening or limit. `check` and the boundary check of a manifest-backed `verify` route a changed selected script as a protected change of the hosts selecting it, from both compared sides' declarations, reading the base's only when the change touches a declaration (`tests/test_hook_script_routing.py`); `check` and a provided diff, which name no limit, leave out a selected script the change does not touch, from their comparison and input coverage alike, and a provided diff that touches both a script and its declaring file compares the script's bytes as the diff states them (`tests/test_hook_script_comparison_limits.py`). | | `application_diff` | `src/agents_shipgate/cli/application_diff.py`, `src/agents_shipgate/cli/application_scope.py`, `src/agents_shipgate/cli/application_summary.py`, `src/agents_shipgate/inputs/tool_reach.py` | — | — | Advisory SDK/ADK source-wiring comparison through `diff --application`. Without `--scope` the scope is derived from the change: the outermost package holding each changed Python file and the agent-building SDK/ADK files (discovery's own signals) that import it or that it imports within six hops, with what they import. Each independent application is its own comparison and never the root by default; agents only the root would join are named as `outside` and make the answer `partial`. A whole application directory moved is one relocation. A change no agent reaches is an explicit `not_established` answer, and a bound reached is named in `scope_selection` and makes the answer `partial` (#875, `tests/test_application_scope.py`). Reuses framework observations and the binding graph; its comparator answers no release or merge verdict, activation verdict, executable pin, or declared authority question. Unlike host mode it computes per-agent source differences, not a projection of host drift. Scoped gaps and `not_established` candidates bound the answer; no deployment-root reachability is asserted. An agent the other side's file still names through a construction no reader supports is a gap there, never a removal or addition, and SDK identity follows the import, so LiveKit's `Agent`/`function_tool` are not read as the SDK's. `tests/test_application_diff.py`, `tests/test_application_diff_review.py` and `tests/test_application_diff_identity.py` prove the advisory boundary, isolation, uncertainty and evidence identity. Every scope, the root included, is materialized by the scoped verified materializer, so a link is recreated rather than refused and never read through; every link under the scope, a dangling one included, is censused and gapped only where it can hide application source (a `*.py` link not aliasing an input the scope reads, a directory link holding Python outside the scope, or an unresolved link where the other side reads source); a submodule is never read, named as a limit when its gitlink commit is unchanged and a coverage gap over its path (the whole scope at the scope itself) otherwise; the host-configuration census adds no application gap (`tests/test_application_diff_reach.py`). A tool an agent binds from another module inside the selected scope is followed by the SDK/ADK readers to its definition, never imported or run, and each module read is published with its digest as `import_path` evidence outside the compared meaning; an import they cannot follow stays a named gap scoped to its agent (#864, `tests/test_imported_tool_bindings.py`). An unobserved agent is not no change (#876): every OpenAI Agents SDK construction in a file the scope reads is an observed agent or a named limit — `return Agent(...)`, `self.agent = Agent(...)`, an inline agent, `Agent[Context](...)` and a positional name are read under their literal `name`; `**` or extra positional arguments, a capability-passing copy, a change to an agent's tools after construction, one identity constructed twice with different tools, a construction without a literal name, and an agent built from an SDK `Agent` subclass or from a class deriving from a Google ADK agent class are limits — so `compared` holds no unaccounted construction site. Test code, by discovery's test-path convention relative to the scope, establishes nothing and is listed per side in the additive `excluded_tests` field, a list of the comparison's own unread input paths that restates no engine answer and adds no claim; a tool name one application file defines twice is a gap on that name, never a refusal (`tests/test_application_diff_unobserved.py`). Each side of a row also carries `reach` (#872). It holds the outbound HTTP calls read statically from the tool's own code and its same-scope helpers, up to three calls deep. For each call it records the method, the URL template and literal request fields, which model-supplied parameters flow where, and the environment variables sent as credentials, by name, never a value. Every call it cannot follow is a named limit. `effect_evidence` is the engine's own `assess_tool_semantics` over that tool, with the reach as one more structural source, `source_http_call`, so it is not a second classifier. `read` is claimed only when every call was followed and every outbound call reads. `reach.effects` (#913) names what the tool's code reaches beyond HTTP through a recognised database, process, file, cloud SDK or messaging library (`src/agents_shipgate/inputs/tool_effects.py`), by import identity and method, never a spelling: a SQL statement, a command's arguments and a withheld path are only digested, and a method outside the tables is a limit. Such an effect is one more structural source of the same assessment, `source_library_call`, and `read` still needs everything reached to read, so it adds no claim either (`tests/test_tool_reach_effects.py`). Both are evidence outside the compared meaning, so they move no row or status. For a Google ADK name constructed more than once, `binding_location` names a construction that lists the tool, and `construction_sites` lists every one (`tests/test_application_diff_tool_reach.py`). A tools, handoffs or sub-agents list built by an expression — a spread, `+`, a conditional, `or`, a filter, another module's list — is read member by member through one resolver both readers share, and each part it cannot read is named, with its location, on the agent it belongs to, which stays incomplete. A member held only under a condition carries `bound_when`, the condition's source text, never evaluated; it restates no engine answer, and a change to it alone is a `changed` row whose direction is not established (#909, `tests/test_list_expressions.py`). A tool bound as an object — an MCP server or toolset, an agent exposed as a tool, a hosted or built-in tool recognised by its import, never its spelling — is a binding whose side carries `object`, what it is (transport, URL host or program file name, credential names, filter; the wrapped agent; the tool name), and `object_evidence`; a URL's path or query, a header value and a command's arguments are only digested, never printed. Both are facts about the source, read for `diff --application` only, so `scan` reads what it did and neither restates an engine answer (#910, `tests/test_object_bound_tools.py`). The comparison opens with an additive `summary` block and the text built from it (#914): one finding per changed agent, the first thing not read, one entry per cause shared by `not_established` rows, and a question specific to what changed. It is a reading of the rows, derived from the sanitized rows and naming the rows it summarizes by index: it moves no row, status, direction, gap, scope or exit code, states nothing a row does not, and is excluded from the benchmark's `answer_id`, so it restates no engine answer and adds no claim. A finding is `partial` when any of its tools has an unread binding or reach, and says so with the first place reading stopped; `comparison_status` is unchanged and still speaks of bindings only (`tests/test_application_summary.py`). | | `zero_install_detector` | `tools/shipgate-detect.py` | `agent_project_verdict` | `test_detector_verdict_matches_cli` | Emits no `diagnostics[]` and no `next_actions[]`; evidence strings and framework scores are simplified. See the script's own "Intentional simplifications". | | `emitted_ci_workflow` | `src/agents_shipgate/cli/discovery/ci_workflow.py` | `executable_pin` | `tests/test_adopter_pins_resolve.py::test_the_emitted_workflow_pins_the_release_and_not_the_source_tree`, `tests/test_release_source.py::test_candidate_workflow_uses_immutable_source_before_and_after_publication` | Ordinary/source/preview builds use the published fallback; a stamped candidate pins its verified Action SHA and package version. Before publication its smoke substitutes the exact local wheel inputs. Provenance asserts no qualification. | diff --git a/docs/host-boundary-support.md b/docs/host-boundary-support.md index 8cc41e95..4ff8c5c7 100644 --- a/docs/host-boundary-support.md +++ b/docs/host-boundary-support.md @@ -101,6 +101,36 @@ into the npm cache only when the project does not depend on it `package.json`, lockfile, `node_modules` or cache is read to decide which applies, and the row's direction, `expands` and severity do not move. + + +An MCP server that passes environment variables through by name declares them +in an `env_vars` list, apart from the `env` map whose keys `env_keys` +publishes. A change confined to that list used to read as `no difference in +the command name …, launch arguments, env key names or header key names; the +change is in a detail this output does not show, such as the command's path or +another setting` (openai/codex-security#1281 added +`CODEX_SECURITY_PLUGIN_ROOT` to it, #795). The grant now publishes +`env_var_names` (host-grants `0.9`), and the row names the names the list +gained and lost, `codex-security: env_vars names +CODEX_SECURITY_PLUGIN_ROOT`, +beside `env keys` and `header keys`: `env keys +API_BASE -DEBUG; env_vars names ++X -Y`. The label is the key the list is declared under, so the row claims no +more for a name than that it is listed there: it does not say a variable is +set, available to the server or read by it, or that either host honours the +field. An entry is published only when it is a plain name, +`[A-Za-z_][A-Za-z0-9_]{0,79}`, that neither the redaction `config_sha256`'s +input applies nor the label redaction rewrites; a `NAME=value` entry, an +object, a token-shaped string and the entry after a credential word such as +`api_key` are not published, and a change confined to them still reads as a +detail this output does not show. The list is digested in order, so the same +names in another order are a row, and it reads `env_vars names in a different +order`. No value is ever published: `env` and `headers` values are redacted +whole in the digest's input, so rotating one is no row, and this entry adds no +per-key value digest. Other fields a host may honour, such as `cwd`, a bearer +token's variable name or a header taken from an environment variable, are not +published. `env_var_names` is display-only: out of grant equality, the +inventory digests and saved baselines, so no row, direction, `expands`, +severity or `check` decision moves. + The boundary is intentionally fail-closed above the adapters' specialized semantics. Most instruction, policy, skill, and workflow edits therefore route to human review unless a dedicated rule can prove the change safe. This can be diff --git a/docs/host-grants-inventory-schema.v0.9.json b/docs/host-grants-inventory-schema.v0.9.json index ef1a0553..660cad33 100644 --- a/docs/host-grants-inventory-schema.v0.9.json +++ b/docs/host-grants-inventory-schema.v0.9.json @@ -1480,6 +1480,13 @@ "title": "Env Keys", "type": "array" }, + "env_var_names": { + "items": { + "type": "string" + }, + "title": "Env Var Names", + "type": "array" + }, "grant_id": { "title": "Grant Id", "type": "string" diff --git a/src/agents_shipgate/core/capability_diff_rows.py b/src/agents_shipgate/core/capability_diff_rows.py index 58461ad2..290d0037 100644 --- a/src/agents_shipgate/core/capability_diff_rows.py +++ b/src/agents_shipgate/core/capability_diff_rows.py @@ -1354,6 +1354,7 @@ def _mcp_cell(value: str, grant: dict[str, Any] | None) -> str: _mcp_launch(grant), f"package {grant['package']}" if grant.get("package") else None, "env keys " + _names(_key_names(grant["env_keys"])) if grant.get("env_keys") else None, + f"{_ENV_VARS_LABEL} " + _names(_key_names(grant["env_var_names"])) if grant.get("env_var_names") else None, "header keys " + _names(_key_names(grant["header_keys"])) if grant.get("header_keys") else None, ) if fact @@ -1395,7 +1396,9 @@ def _mcp_change(name: str, before: dict[str, Any], after: dict[str, Any]) -> str confined to them says what was compared and that the change is elsewhere, rather than ``name → name`` or a claim that the command is unchanged. Two different endpoints that print alike, such as two URLs - neither of which is printed, read ``url changed (not shown)``. + neither of which is printed, read ``url changed (not shown)``. The names + an ``env_vars`` list gained and lost are named when both readings publish + them (#795); a value is never read. """ if any(key not in grant for grant in (before, after) for key in _MCP_FIELDS): @@ -1417,19 +1420,62 @@ def _mcp_change(name: str, before: dict[str, Any], after: dict[str, Any]) -> str kind = "url" if after.get("transport") == "url" else "command name" parts.append(f"{kind} changed ({_URL_NOT_SHOWN})") parts.extend(_mcp_args_change(before, after)) - for field, label in (("env_keys", "env keys"), ("header_keys", "header keys")): - old, new = set(before[field] or []), set(after[field] or []) - added, removed = sorted(new - old), sorted(old - new) - if added or removed: - tokens = [f"+{key}" for key in _key_names(added)] + [f"-{key}" for key in _key_names(removed)] - parts.append(f"{label} {_names(tokens)}") + parts.extend(_key_set_change(before, after, "env_keys", "env keys")) + env_vars = _env_vars_compared(before, after) + if env_vars: + parts.extend(_key_set_change(before, after, "env_var_names", _ENV_VARS_LABEL, ordered=True)) + parts.extend(_key_set_change(before, after, "header_keys", "header keys")) if not parts: compared = all("args_sha256" in grant for grant in (before, after)) - return f"{name}: {_mcp_unshown_change(after, args_compared=compared)}" + return f"{name}: {_mcp_unshown_change(after, args_compared=compared, env_vars=env_vars)}" return f"{name}: " + "; ".join(parts) -def _mcp_unshown_change(grant: dict[str, Any], *, args_compared: bool = False) -> str: +#: What a row calls a server's ``env_vars`` list: the key it is declared under, +#: so no meaning is claimed for a name beyond that it is listed there (#795). +_ENV_VARS_LABEL = "env_vars names" + + +def _env_vars_compared(before: dict[str, Any], after: dict[str, Any]) -> bool: + """Whether both readings publish ``env_var_names`` and either declares one. + + A saved baseline, or a grant from an earlier schema, publishes none, so + nothing is claimed about the names it cannot show. Two readings that + declare none have nothing to say about them. + """ + + return all("env_var_names" in grant for grant in (before, after)) and bool( + before["env_var_names"] or after["env_var_names"] + ) + + +def _key_set_change( + before: dict[str, Any], after: dict[str, Any], field: str, label: str, *, ordered: bool = False +) -> list[str]: + """The names a published list gained and lost, never a value (#795). + + Names print through the #802 label redaction. ``ordered`` is for a list + whose order is part of its declaration (``env_vars``): the same names in + another order are said to be, rather than called no difference. + """ + + old_list, new_list = list(before[field] or []), list(after[field] or []) + old, new = set(old_list), set(new_list) + added, removed = sorted(new - old), sorted(old - new) + if added or removed: + tokens = [f"+{key}" for key in _key_names(added)] + [f"-{key}" for key in _key_names(removed)] + return [f"{label} {_names(tokens)}"] + if ordered and old_list != new_list: + # The same set of names: the declaration still differs, in its order + # or in how many times a name is listed. + same = sorted(old_list) == sorted(new_list) + return [f"{label} in a different order" if same else f"{label} listed a different number of times"] + return [] + + +def _mcp_unshown_change( + grant: dict[str, Any], *, args_compared: bool = False, env_vars: bool = False +) -> str: """A change confined to what the grant does not publish, in the words of what was compared. Only the command's name, or the URL's recorded value, the launch @@ -1457,8 +1503,9 @@ def _mcp_unshown_change(grant: dict[str, Any], *, args_compared: bool = False) - else: compared = launch or "command name" unshown = "the command's path or another setting" if arguments else "the command's path or arguments" + names = f"env key names, {_ENV_VARS_LABEL} or header key names" if env_vars else "env key names or header key names" return ( - f"no difference in the {compared}, {arguments}env key names or header key names; the " + f"no difference in the {compared}, {arguments}{names}; the " f"change is in a detail this output does not show, such as {unshown}" ) diff --git a/src/agents_shipgate/core/host_grants.py b/src/agents_shipgate/core/host_grants.py index 301ca894..29f88076 100644 --- a/src/agents_shipgate/core/host_grants.py +++ b/src/agents_shipgate/core/host_grants.py @@ -1259,6 +1259,40 @@ def _mcp_launch_args(config: dict[str, Any]) -> tuple[str | None, str | None]: return None, redacted_config_sha256(args) +#: A name ``env_vars`` may publish: the shape of an environment variable's name +#: and no more, so a ``NAME=value`` entry or any other text is never published +#: (#795). +_ENV_VAR_NAME = re.compile(r"[A-Za-z_][A-Za-z0-9_]{0,79}") + + +def _mcp_env_var_names(config: dict[str, Any]) -> list[str]: + """The names an MCP server's ``env_vars`` list declares, as written (#795). + + ``env_keys`` publishes the keys of the ``env`` map; a server that passes + variables through by name declares them in an ``env_vars`` list instead, + which the key names never held. A name is published only when it is a + plain environment-variable name that neither the redaction + ``config_sha256``'s input applies (:func:`_redact_secret_values`) nor the + label redaction (:func:`published_workflow_label`) rewrites, so a value, a + ``NAME=value`` entry, an object, a token-shaped string or the entry a + credential word before it makes the digest redact is not. Their declared + order is kept, so an edit that only reorders the list can be told from an + edit to its names. Display-only: the list is already in + ``config_sha256``'s input, which is where a change to any entry, published + or not, becomes a row. + """ + + declared = config.get("env_vars") + if not isinstance(declared, list): + return [] + as_digested = _redact_secret_values(declared, parent_key="env_vars") + return [ + item for item, digested in zip(declared, as_digested, strict=True) + if isinstance(item, str) and item == digested and _ENV_VAR_NAME.fullmatch(item) + and published_workflow_label(item) == item + ] + + def _mcp_launch_source(config: dict[str, Any]) -> dict[str, Any] | None: if _transport_hint(config) != "stdio" or "url" in config: return None @@ -1374,6 +1408,7 @@ def _mcp_grants( "endpoint": _endpoint(config), "env_keys": sorted(str(key) for key in env), "header_keys": sorted(str(key) for key in headers), + "env_var_names": _mcp_env_var_names(config), "package": package, "args_sha256": args_sha256, "launch_source": _mcp_launch_source(config), @@ -7007,7 +7042,7 @@ def diff_host_grants(baseline: dict[str, Any], current: dict[str, Any]) -> list[ #: (:func:`build_host_grants_baseline`). DISPLAY_ONLY_GRANT_FIELDS: dict[str, frozenset[str]] = { "hook": frozenset({"handlers", "omitted_handlers"}), - "mcp_server": frozenset({"package", "args_sha256", "launch_source"}), + "mcp_server": frozenset({"package", "args_sha256", "launch_source", "env_var_names"}), } diff --git a/src/agents_shipgate/schemas/host_grants.py b/src/agents_shipgate/schemas/host_grants.py index 9f87c586..9723cc00 100644 --- a/src/agents_shipgate/schemas/host_grants.py +++ b/src/agents_shipgate/schemas/host_grants.py @@ -1235,6 +1235,14 @@ class HostMcpServerGrantV9(HostMcpServerGrantV7): host: McpHostNameV9 launch_source: HostMcpLaunchSourceV9 | None = None + #: The plain environment-variable names the server's ``env_vars`` list + #: declares, in declared order (#795): names only, never a value, and none + #: that the redaction ``config_sha256``'s input applies or the label + #: redaction would rewrite. An entry that is not such a name is not + #: published. Display-only, left out of grant equality, the inventory + #: digests and saved baselines, like ``package``. Extended in place: ``0.9`` + #: has not shipped in a tagged release. + env_var_names: list[str] = Field(default_factory=list) class HostMcpServerBaselineGrantV9(HostMcpServerGrantV2): diff --git a/tests/shard_seconds.json b/tests/shard_seconds.json index 25002195..a71674ff 100644 --- a/tests/shard_seconds.json +++ b/tests/shard_seconds.json @@ -241,6 +241,7 @@ "tests/test_manifest_schema_parity.py": 0.0, "tests/test_manifest_scope.py": 0.0, "tests/test_mcp_audit.py": 0.1, + "tests/test_mcp_env_var_names.py": 60.0, "tests/test_mcp_host_attribution.py": 48.5, "tests/test_mcp_idioms.py": 0.0, "tests/test_mcp_launch_source.py": 43.5, diff --git a/tests/test_distribution_surface_parity.py b/tests/test_distribution_surface_parity.py index bd268b86..d7a67394 100644 --- a/tests/test_distribution_surface_parity.py +++ b/tests/test_distribution_surface_parity.py @@ -365,6 +365,12 @@ def paths(self) -> list[Path]: # neither adds a claim or moves a row, severity or `check` decision; # `tests/test_mcp_row_bounded_wording.py` holds diff, verify, the PR # comment and both `check` routes to them. + # An MCP row's `env_vars` names (#795) render the display-only + # `env_var_names` the engine published on the grant, plain names only + # and never a value, left out of grant equality, the inventory digests + # and saved baselines, so they restate no answer and move no row, + # severity or `check` decision; `tests/test_mcp_env_var_names.py` holds + # diff, verify, the PR comment and check to the same entry. # An MCP row's subject and `why` naming the host whose declaration # selects its `.mcp.json`, or no host for one the inventory published # under `unknown` (#936), restate the inventory's attribution, the diff --git a/tests/test_mcp_env_var_names.py b/tests/test_mcp_env_var_names.py new file mode 100644 index 00000000..8d4f3576 --- /dev/null +++ b/tests/test_mcp_env_var_names.py @@ -0,0 +1,347 @@ +"""#795: an MCP row names which `env_vars` entries changed, and never a value. + +Replaying openai/codex-security#1281, `plugins/codex-security/.mcp.json` added +`CODEX_SECURITY_PLUGIN_ROOT` to a server's `env_vars` array and changed +nothing else. The row read `no difference in the command name …, launch +arguments, env key names or header key names; the change is in a detail this +output does not show`, because the grant published the keys of the `env` map +and not the names an `env_vars` list passes through. The grant now publishes +those names (host-grants `0.9`, extended in place) and the row names the ones +that moved, beside the existing `env keys` and `header keys`. + +Display only: no row, direction, `expands`, severity, expansion signal, +digest, baseline or `check` decision moves. The `env` map's values are +redacted whole in `config_sha256`'s input, so rotating one is still no row, +and no value is published or printed. +""" + +from __future__ import annotations + +import json +import re +from pathlib import Path + +import pytest + +from agents_shipgate.core.capability_diff_rows import ( + _mcp_change, + capability_diff_rows, + review_changes, +) +from agents_shipgate.core.host_grants import ( + HostStaticParseCache, + _mcp_env_var_names, + _mcp_grants, + build_host_boundary_snapshot, + build_host_drift_payload, + build_host_grants_baseline, + compared_grant, + host_grants_sha256, +) +from agents_shipgate.schemas.host_grants import HostMcpServerBaselineGrantV9, HostMcpServerGrantV9 +from tests.test_hook_mcp_detail_fields import _boundary, _every_route +from tests.test_host_diff_review_changes import ( + _check, + _diff, + _git, + _repository, + _table_entry, + _verify, + _write, +) + +PLUGIN_MCP = "plugins/codex-security/.mcp.json" +HEADER = f"high changed claude-code {PLUGIN_MCP}" +SERVER = { + "command": "launch_codex_security_mcp", + "args": ["--stdio"], + "env": {"CODEX_HOME": "canary-home-value"}, +} +SHOWN = ( + "no difference in the command name launch_codex_security_mcp, launch arguments, " + "env key names, env_vars names or header key names; the change is in a detail this output " + "does not show, such as the command's path or another setting" +) + + +def _mcp(**fields: object) -> dict: + return {"mcpServers": {"codex-security": {**SERVER, **fields}}} + + +def _rows(payload: dict) -> list[dict]: + """The published rows, and the presentation of each but its text.""" + + return [ + *payload["rows"], + *({key: value for key, value in entry.items() if key != "change"} for entry in payload["review"]["changes"]), + ] + + +# --- the reproduction, on every route --------------------------------------- + + +def test_the_reproduction_names_the_added_env_var_on_every_route(tmp_path: Path) -> None: + repo = _repository( + tmp_path, {PLUGIN_MCP: _mcp()}, {PLUGIN_MCP: _mcp(env_vars=["CODEX_SECURITY_PLUGIN_ROOT"])} + ) + change = "codex-security: env_vars names +CODEX_SECURITY_PLUGIN_ROOT" + + text, payload = _diff(repo) + assert _table_entry(text, HEADER) == [ + HEADER, change, "MCP edit; authority direction is unknown", + ] + _every_route(repo, tmp_path / "out", change) + # `check --diff` reads the same rows from a patch. + patch = tmp_path / "change.diff" + patch.write_text(_git(repo, "diff", "main", "HEAD") + "\n", encoding="utf-8") + assert f" {change}" in _check(repo, "--diff", str(patch)) + [row] = payload["rows"] + assert (row["direction"], row["expands"], row["severity"]) == ("changed", False, "high") + assert "this output does not show" not in text + assert "canary-home-value" not in text + json.dumps(payload) + + +@pytest.mark.parametrize( + "base,head,change", + [ + pytest.param(["A"], ["A", "B"], "env_vars names +B", id="added"), + pytest.param(["A", "B"], ["A"], "env_vars names -B", id="removed"), + pytest.param(["A", "B"], ["B", "C"], "env_vars names +C -A", id="both"), + pytest.param(["A", "B"], ["B", "A"], "env_vars names in a different order", id="reordered"), + pytest.param(["A"], ["A", "A"], "env_vars names listed a different number of times", id="repeated"), + pytest.param( + ["NPM_TOKEN"], ["NPM_TOKEN", "GITHUB_TOKEN"], "env_vars names +GITHUB_TOKEN", + id="a-name-with-a-credential-word", + ), + pytest.param( + [f"N{index}" for index in range(8)], [f"N{index}" for index in range(2, 10)], + "env_vars names +N8 +N9 -N0 -N1", id="several", + ), + pytest.param( + [], [f"N{index}" for index in range(7)], + "env_vars names +N0 +N1 +N2 +N3 +N4 and 2 more", id="bounded", + ), + ], +) +def test_each_change_to_the_list_is_named(base: list, head: list, change: str) -> None: + assert _mcp_change("docs", _grant(env_vars=base), _grant(env_vars=head)) == f"docs: {change}" + + +def test_a_reorder_is_a_row_on_every_route(tmp_path: Path) -> None: + """`config_sha256` reads the list in order, so a reorder is a row, and it says only that.""" + + repo = _repository( + tmp_path, {PLUGIN_MCP: _mcp(env_vars=["A", "B"])}, {PLUGIN_MCP: _mcp(env_vars=["B", "A"])} + ) + _every_route(repo, tmp_path / "out", "codex-security: env_vars names in a different order") + + +def test_the_list_is_named_beside_the_env_map_and_the_headers(tmp_path: Path) -> None: + base = {"mcpServers": {"remote": { + "url": "https://mcp.example.com/path", "env": {"DEBUG": "1"}, "headers": {"X-Old": "o"}, + "env_vars": ["A"], + }}} + head = {"mcpServers": {"remote": { + "url": "https://mcp.example.com/path", "env": {"API_BASE": "1"}, "headers": {"X-New": "n"}, + "env_vars": ["B"], + }}} + repo = _repository(tmp_path, {".mcp.json": base}, {".mcp.json": head}) + _every_route( + repo, tmp_path / "out", + "remote: env keys +API_BASE -DEBUG; env_vars names +B -A; header keys +X-New -X-Old", + ) + + +def test_an_added_and_a_removed_server_list_the_names_they_pass_through(tmp_path: Path) -> None: + (tmp_path / "added").mkdir() + (tmp_path / "removed").mkdir() + server = {"command": "npx", "args": ["-y", "example-mcp-server@2.0.0"], "env_vars": ["A", "B"]} + added = _repository(tmp_path / "added", {".mcp.json": {"mcpServers": {}}}, {".mcp.json": {"mcpServers": {"docs": server}}}) + text, payload = _diff(added) + assert "docs (command name npx; package example-mcp-server@2.0.0; env_vars names A B)" in " ".join(text.split()) + cell = "docs (command name npx; package example-mcp-server@2.0.0; env_vars names A B)" + assert payload["review"]["changes"][0]["after"] == cell + removed = _repository( + tmp_path / "removed", {".mcp.json": {"mcpServers": {"docs": server}}}, {".mcp.json": {"mcpServers": {}}} + ) + assert _diff(removed)[1]["review"]["changes"][0]["before"] == cell + + +def test_a_codex_config_names_its_env_vars(tmp_path: Path) -> None: + def config(names: str) -> str: + return f'[mcp_servers.docs]\ncommand = "npx"\nargs = ["-y", "example-mcp-server@1.2.3"]\nenv_vars = {names}\n' + + repo = _repository( + tmp_path, {".codex/config.toml": config('["A"]')}, {".codex/config.toml": config('["A", "B"]')} + ) + text, payload = _diff(repo) + assert [entry["change"] for entry in payload["review"]["changes"]] == ["docs: env_vars names +B"] + assert "env_vars names +B" in text + + +# --- what stays as it was ---------------------------------------------------- + + +def test_nothing_but_the_text_moves(tmp_path: Path) -> None: + """A head the names cannot describe is the same row and `check` result, with the old wording.""" + + (tmp_path / "named").mkdir() + (tmp_path / "unnamed").mkdir() + named = _repository(tmp_path / "named", {PLUGIN_MCP: _mcp()}, {PLUGIN_MCP: _mcp(env_vars=["NAME"])}) + # An object is not a plain name, so it is not published; the list still changes the digest. + unnamed = _repository( + tmp_path / "unnamed", {PLUGIN_MCP: _mcp()}, {PLUGIN_MCP: _mcp(env_vars=[{"name": "NAME"}])} + ) + named_payload, unnamed_payload = _diff(named)[1], _diff(unnamed)[1] + assert _rows(named_payload) == _rows(unnamed_payload) + assert named_payload["review"]["changes"][0]["change"] == "codex-security: env_vars names +NAME" + assert unnamed_payload["review"]["changes"][0]["change"].startswith("codex-security: no difference in") + + def boundary(repo: Path) -> str: + """`check`'s whole result, with the one repository's path and head commit set aside.""" + + return re.sub( + r"boundary_[0-9a-f]{24}", "boundary_", + json.dumps(_boundary(repo), sort_keys=True) + .replace(str(repo), "") + .replace(_git(repo, "rev-parse", "HEAD"), ""), + ) + + assert boundary(named) == boundary(unnamed) + + +def test_a_value_the_digest_redacts_is_still_no_row(tmp_path: Path) -> None: + """`env` values are redacted whole in `config_sha256`'s input: a rotation is no change.""" + + repo = _repository( + tmp_path, + {PLUGIN_MCP: _mcp(env={"API_TOKEN": "canary-first-value"})}, + {PLUGIN_MCP: _mcp(env={"API_TOKEN": "canary-second-value"})}, + ) + text, payload = _diff(repo) + assert payload["rows"] == [] and payload["review"]["changes"] == [] + assert "canary" not in text + json.dumps(payload) + + +def test_the_same_env_map_in_another_order_is_no_row(tmp_path: Path) -> None: + repo = _repository( + tmp_path, + {PLUGIN_MCP: _mcp(env={"A": "1", "B": "2"}, env_vars=["X", "Y"])}, + {PLUGIN_MCP: _mcp(env={"B": "2", "A": "1"}, env_vars=["X", "Y"])}, + ) + assert _diff(repo)[1]["rows"] == [] + + +def test_an_unpublished_entry_keeps_the_wording_for_what_is_not_shown(tmp_path: Path) -> None: + """A `NAME=value` entry, an object or a token-shaped name is never published, and the row says so.""" + + token = "ghp_" + "Z9y8X7w6V5u4T3s2R1q0P9o8N7m6L5k4J3i2" + entries = [f"API_BASE=canary-{token}", {"name": "OBJ", "source": "canary-object"}, token, "with space"] + repo = _repository( + tmp_path, {PLUGIN_MCP: _mcp(env_vars=["A"])}, {PLUGIN_MCP: _mcp(env_vars=["A", *entries])} + ) + text, payload = _diff(repo) + [entry] = payload["review"]["changes"] + assert entry["change"] == f"codex-security: {SHOWN}" + block, _summary, verifier = _verify(repo, tmp_path / "out") + outputs = [ + text, json.dumps(payload), "\n".join(block), json.dumps(verifier), + (tmp_path / "out" / "pr-comment.md").read_text(encoding="utf-8"), + "\n".join(_check(repo)), json.dumps(_boundary(repo)), + ] + for output in outputs: + assert "canary" not in output and token not in output and "OBJ" not in output + + +def test_the_published_names_are_plain_names_the_digest_keeps() -> None: + token = "ghp_" + "Z9y8X7w6V5u4T3s2R1q0P9o8N7m6L5k4J3i2" + names = [ + "PLAIN", "_UNDER", "lower_case", "GITHUB_TOKEN", "A" * 80, + "A" * 81, "1LEADING_DIGIT", "NAME=value", "has-dash", "", token, 7, None, {"name": "OBJ"}, + ] + assert _mcp_env_var_names({"env_vars": names}) == [ + "PLAIN", "_UNDER", "lower_case", "GITHUB_TOKEN", "A" * 80, + ] + # The entry a credential word before it makes the digest redact is not published either. + assert _mcp_env_var_names({"env_vars": ["A", "api_key", "B", "C"]}) == ["A", "api_key", "C"] + for declared in (None, "NAME", {"NAME": "value"}, 3): + assert _mcp_env_var_names({"env_vars": declared}) == [] + assert _mcp_env_var_names({}) == [] + + +def test_a_credential_word_before_a_name_hides_a_change_the_digest_never_saw(tmp_path: Path) -> None: + repo = _repository( + tmp_path, + {PLUGIN_MCP: _mcp(env_vars=["API_KEY", "ONE"])}, + {PLUGIN_MCP: _mcp(env_vars=["API_KEY", "TWO"])}, + ) + assert _diff(repo)[1]["rows"] == [] + + +# --- the grant ---------------------------------------------------------------- + + +def _grant(**fields: object) -> dict: + (grant,) = _mcp_grants( + {"mcpServers": {"docs": {**SERVER, **fields}}}, host="claude-code", scope="repository", source=".mcp.json" + ) + return grant + + +def test_the_field_is_display_only() -> None: + plain, named = _grant(), _grant(env_vars=["A", "B"]) + assert plain["env_var_names"] == [] and named["env_var_names"] == ["A", "B"] + HostMcpServerGrantV9.model_validate(named) + # The list is in the digest's input, so a change to it is still a row; the published names are not compared. + assert plain["config_sha256"] != named["config_sha256"] + assert compared_grant(named) == compared_grant({**named, "env_var_names": ["Z"]}) + assert "env_var_names" not in compared_grant(named) + + def inventory(grant: dict) -> dict: + return {"grants": [grant]} + + other = {**named, "env_var_names": ["Z"]} + assert host_grants_sha256(inventory(named)) == host_grants_sha256(inventory(other)) + + +def test_a_saved_baseline_holds_no_names(tmp_path: Path) -> None: + root = tmp_path / "repo" + _write(root, ".mcp.json", _mcp(env_vars=["A", "B"])) + inventory = build_host_boundary_snapshot(root, cache=HostStaticParseCache()).inventory + (grant,) = [grant for grant in inventory["grants"] if grant["kind"] == "mcp_server"] + assert grant["env_var_names"] == ["A", "B"] + baseline = build_host_grants_baseline(inventory) + (saved,) = [grant for grant in baseline["inventory"]["grants"] if grant["kind"] == "mcp_server"] + assert "env_var_names" not in saved + HostMcpServerBaselineGrantV9.model_validate(saved) + # The names are not in the digest: another list of them gives the same one. + other = { + **inventory, + "grants": [{**g, "env_var_names": ["Z"]} if g["kind"] == "mcp_server" else g for g in inventory["grants"]], + } + assert build_host_grants_baseline(other)["inventory_sha256"] == baseline["inventory_sha256"] + assert host_grants_sha256(other) == host_grants_sha256(inventory) + + drift = build_host_drift_payload(baseline=baseline, inventory=inventory, baseline_file="b.json") + assert (drift["comparison_status"], drift["has_drift"], drift["changes"]) == ("comparable", False, []) + # A change is still a row; the saved side names no list it cannot show. + _write(root, ".mcp.json", _mcp(env_vars=["A", "C"])) + changed = build_host_drift_payload( + baseline=baseline, inventory=build_host_boundary_snapshot(root, cache=HostStaticParseCache()).inventory, + baseline_file="b.json", + ) + [row] = capability_diff_rows(changed) + [presented] = review_changes([row]) + assert "env_vars names" not in (presented.change or "") + + +def test_a_reading_that_publishes_no_names_claims_nothing_about_them() -> None: + base, head = _grant(env_vars=["A"]), _grant(env_vars=["A", "B"]) + legacy = {key: value for key, value in base.items() if key != "env_var_names"} + # A baseline's side has none of the display members; the other side's names are not read as a change. + assert "env_vars names" not in (_mcp_change("docs", legacy, head) or "") + assert _mcp_change("docs", base, head) == "docs: env_vars names +B" + assert _mcp_change("docs", base, base) == f"docs: {SHOWN}" + # Neither side declaring any leaves the sentence as it was. + unchanged = _mcp_change("docs", _grant(), _grant()) + assert unchanged is not None and "env_vars names" not in unchanged