Repository navigation
Conversation
…lose
Every policy reload advances one sandbox-wide generation, and every pinned
relay closes when it moves, so approving an endpoint for one host drops
every open proxied connection in the sandbox.
Add tests, against the existing relay and engine entry points, for how
established connections should respond to reloads. Each test's doc comment
says whether it describes behaviour the following change alters
("Changed", failing on this commit) or behaviour it must keep
("Unchanged", passing before and after).
Changed: connections stay open when a reload does not affect them:
- raw TCP, HTTP passthrough and inspected (L7) relays, when an endpoint is
approved for another host, an endpoint is inserted before theirs, or a
gateway-delivered policy is restamped with a new policy hash;
- relays without middleware, when middleware is added for another host or
the middleware registry is replaced; a kept-alive passthrough connection
that runs middleware, on its next request after an unrelated reload;
- relays that sleep through several unrelated reloads, or are checked
between any number of them;
- HTTP relays, across provider credential changes they did not take part
in: credentials never injected on the connection, other providers'
rotations and attachments (including through install_prepared), identical
reinstalls, suppression of a credential from child processes, and a
credential first injected after its provider was attached or served
through an aged-out placeholder, and a credential sent through its
canonical or stable-handle placeholder, or through the previous
revision's placeholder while unchanged, when another credential rotates;
- a kept-alive MCP tunnel keeps reporting endpoint status after an
unrelated reload, against the new policy once its inventory is published.
Unchanged: connections still close when a reload does affect them:
revoked endpoints, narrowed L7 rules (YAML and gateway-delivered),
middleware added for their host, changed or made fail-closed, a replaced
registry for connections that run middleware, reloads that revoked or
changed them even if a later reload restored them, a long run of reloads
they slept through, rotated or detached credentials they injected
(including rotations that coalesced, and a rotation behind an unchanged
stable-handle placeholder), a superseded value sent through an older
retained revision, untracked credential handles, a reload between
authorization and relay setup, and plain generation guards.
Endpoint-status results of a kept-alive MCP tunnel are never attributed to
an inventory of a policy it is not running under: not before the reloaded
policy's inventory is published, not to an inventory published ahead of the
runtime, and not after a rules change, which closes the tunnel.
One test records that every reload restamps policy_hash on unchanged
endpoints.
Changed, in the supervisor: revoking provider credentials (an unavailable
environment, bindings that fail validation, or a snapshot for another
revision) publishes a policy generation, so open connections re-check the
credentials they sent. Today revocation publishes nothing, and a
connection authenticated once with a revoked credential, such as an
upgraded WebSocket, stays open.
Unchanged: a revocation that finds nothing left to revoke, as when a
failed refresh repeats on every poll, publishes no generation.
Existing relay tests use Arc<OpaEngine>, which the change requires.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… affect them Every policy reload, middleware registry change and provider credential change advanced one sandbox-wide generation, and every pinned relay closed when it moved. Approving an endpoint for one host dropped every open proxied connection in the sandbox. When the generation moves, a pinned guard now re-checks the connection it protects and closes only if the answer changed. Policy: a ConnectionPolicyContract captures what decides how a connection is relayed: the L4 action and matched policy, the ordered matched endpoints (policy name and endpoint object, ignoring the endpoint index and the per-reload policy_hash stamp), the exact-host flag, and the host's middleware chain plus the registry epoch when that chain is non-empty. Endpoint configs, which relays also read, are not compared separately: Rego derives them from the same endpoints, in the same order. The engine records its last 16 published generations, and a waking connection is re-checked against every generation since its pin, because the generation watch coalesces reloads. If every contract equals the one that authorized it, the guard re-pins; any difference, evaluation error, fail-closed generation, or pin that has left the history makes the guard permanently stale and the relay closes as before. Per-request Rego rules read only the matched endpoints, so a tunnel's cloned evaluator stays valid after a re-pin. Credentials: the provider credential state owns one version per credential (static environment key or token-grant provider:credential identity), recomputed lazily by comparing content whenever its snapshot changes. Each proxied connection gets a tracked handle and records the version of every credential it injects, at the two points credential material leaves the supervisor: SecretResolver handing out a secret, and inject_or_report adding a grant. HTTP-family relays close only when a credential they injected changes or is removed, or when they sent a value that was already superseded. A use counts as current when the value it handed out is its key's current value, whichever placeholder form named it: provider revisions are sandbox-wide, so an older revision's placeholder usually still names the current value. Untracked handles and unattributable uses close on the next check. Raw TCP and TLS passthrough relays never inject credentials and carry no pin. Endpoint status: a tunnel's endpoint config carries the hash of the policy it was selected from, so an MCP tunnel that re-pinned would have stopped reporting endpoint status. The revalidator now records the stamped policy hash of the generation its guard is pinned to, from authorization through every re-pin, and observations (and token-grant reports) use it. The inventory check is unchanged, so results are still dropped while the inventory describes a different policy. Revocation: the supervisor revokes static provider credentials when a provider refresh fails, its bindings fail validation, or it delivers the wrong revision, but it published no generation, so connections that had sent a revoked credential were never checked. Both revocation paths now call OpaEngine::publish_credential_change, which publishes a generation of the unchanged policy (keeping any quarantine), when the revocation changed any credential, judged by comparing ProviderCredentialState::installed_credential_versions before and after. A failed refresh that repeats on every poll therefore wakes connections once, not every time. Connections that sent the revoked credential close; the rest re-pin. A reload between authorization and relay setup still refuses the relay. OpaEngine::revalidating_generation_guard is the single entry point for pinning a revalidating guard, and it and with_revalidator are crate-private, so every guard another crate obtains from the engine is plain: checking it is a cheap comparison that never evaluates policy or takes the engine lock. The HTTP passthrough middleware path lets the guard re-pin before comparing raw generation numbers. The policy docs now describe which changes close a connection. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Unit tests for the code the previous commit introduces, which need its new APIs. Behaviour visible through the relays is covered by the earlier characterisation tests. - contracts: unrelated endpoints are ignored; endpoint, L7 rule, binary and overlapping-policy changes close the connection and keep it closed after the policy is restored; quarantine always closes; policy_hash restamping is not a change; denials capture no revalidator; guard clones share one pin; the wait ignores unrelated reloads; - middleware: any field of the selected chain changing closes the connection, an identical chain does not, and only reloads that replace the registry close connections that run middleware; - capture: baselines come from the recorded authorizing generation, and a generation that left the history cannot be captured; - credential versions: recording of key and version per secret sent, including SigV4-style lookups; unused credentials; versions shared across connections and stable across identical reinstalls; a removed and restored key gets a new version; an older revision records stale only if its value has since changed; child-env-only installs and revocation retire credentials; grant versions follow their configuration; untracked handles and unattributable uses; - installed credential versions: unchanged by identical reinstalls and by a revocation with nothing left to revoke; changed by a new value, an added or removed credential, and a grant configuration change; - credential change publication: a new generation of the unchanged policy makes plain guards stale and lets a connection without credentials re-pin; a quarantine stays in force; - pinned policy hash: the authorizing generation's from the start, then that of the generation the guard re-pinned to (not a newer one it has not checked); none for plain guards or policies without a stamp; - endpoint configs are exactly the matched endpoints that carry extended configuration, in the same order, which is why the contract does not compare them; - token grants record their identity; passthrough middleware runs on the next request after an unrelated reload. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Test-only. The first commit's tests were written against the APIs that existed before the change so that they could run, and fail, on their own. Now that the change has landed: - authorize test decisions through EgressDecision::policy_input, the input revalidation re-evaluates; - check that injecting a credential is recorded on the connection, and that a connection that never injected one records nothing; - drop the test module's own ProviderCredentialState import, which the module now inherits; - drop the "Changed"/"Unchanged" labels and the descriptions of the old behaviour from the doc comments. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thank you for your submission! We ask that you sign our Developer Certificate of Origin before we can accept your contribution. You can sign the DCO by adding a comment below using this text: I have read the DCO document and I hereby sign the DCO. You can retrigger this bot by commenting recheck in this Pull Request. Posted by the DCO Assistant Lite bot. |
Author
|
I have read the DCO document and I hereby sign the DCO. |
mr-miles
changed the title
Author
|
recheck |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every network policy reload, provider attach/detach, and middleware registry change causes outbound network connections to be closed by the supervisor, and errors reported back to the sandbox.
This was deliberate (#2373, "invalidate relays on policy changes") and is documented, but much coarser than it needs to be. Also the behaviour is very confusing for workloads that ask for approvals while they are running, causing agents to focus on "solving" and diagnosing connectivity issues rather than their assigned work.
This PR adds the ability to re-validate open connections against a newly received policy, and close them only if the new policy would not allow them. The same behaviour applies for credential changes.
Related issue
Changes
Policy reloads
Provider credentials
Credential revocation (pre-existing bug)
Endpoint status (MCP)
Safety boundaries
Release notes
Re-validate open connections against a newly received policy, and close them only if the new policy would not allow them
Testing
Structured as red/green refactors
Manual check: open a long-lived connection (for example a WebSocket or
curlto a slow endpoint) through the sandbox proxy, approve an endpoint for a different host, and confirm that the connection survives, the log shows "Policy change does not affect established connection; re-pinned", and the connection closes when its own host's rules change.Checklist