diff --git a/docs/evidence/g01-broker.md b/docs/evidence/g01-broker.md index 79250a58..a176be48 100644 --- a/docs/evidence/g01-broker.md +++ b/docs/evidence/g01-broker.md @@ -28,14 +28,16 @@ Runner-group and private repository inspection require an installation/user auth 0. Validate private inputs and the supported native controller build; reserve, write and sync the exact non-secret controller snapshot when applicable. Acquire the fixed account ledger, pin resources/controller state, consume this phase slot durably, and check compatible existing controller inventory. For controller mode, run the credential-free canonical local-state preparation and validate its exact receipt before any authenticated request. These local checks precede every API call. 1. Verify App, owner and organization installation with the App JWT. 2. Flush a non-secret token issuance intent. -3. Flush the non-secret read-token intent, then request an installation token with repository access restricted to `repository_ids: [exact_canary_id]` and exactly `organization_self_hosted_runners: read` plus `metadata: read`. The organization-only scope requests only `organization_self_hosted_runners: read` and no repository IDs. -4. Validate the returned read-token scope and expiry, then use it to verify the exact private non-fork repository (for repository scope), the named nondefault/noninherited runner group, public repositories explicitly disallowed, selected-only visibility, exact workflow restriction and the one selected repository. A failed check stops before any write token is requested. This first preflight is read-only. -5. After read-only preflight succeeds, repeat the final reusable-callee workflow GET. Only then request the management installation token with the exact existing write scope. Validate its returned permission/repository scope and repeat provider-state preflight with that token. Then repeat the same workflow GET immediately before handoff. A mismatch at either later check refuses child handoff, but any already-issued token may remain valid until expiry. +3. Flush the non-secret read-token intent, then immediately before dispatching the installation-token POST, sync `read_token_lifecycle_started` to the fixed account admission ledger. If that sync cannot be established, do not send the request. When a v3 attempt exits before this boundary, sync `read_token_request_not_dispatched`; this preserves later recovery without claiming a request happened. Request a token restricted to `repository_ids: [exact_canary_id]` and exactly `organization_self_hosted_runners: read` plus `metadata: read`. The organization-only scope requests only `organization_self_hosted_runners: read` and no repository IDs. If the response exposes a syntactically valid token but fails typed decoding, expiry or exact-scope validation, do not use it for provider reads; make one self-revocation attempt under the already-durable lifecycle intent. If the returned read token equals the caller workflow-verification token, record the collision and stop before provider-state preflight or DELETE, leaving the lifecycle unresolved. +4. Validate the accepted read-token scope and expiry, then use it to verify the exact private non-fork repository (for repository scope), the named nondefault/noninherited runner group, public repositories explicitly disallowed, selected-only visibility, exact workflow restriction and the one selected repository. After this preflight attempt, durably record a revocation intent and revoke the broker-issued read token with `DELETE /installation/token`. Only `204` with an empty response confirms revocation. An unsuccessful or ambiguous result is best-effort recorded and stops before the final workflow check or any write token request. This first provider preflight is read-only; token revocation is a separate credential effect. +5. After the read token is confirmed revoked and the read-only preflight succeeded, repeat the final reusable-callee workflow GET. Only then request the management installation token with the exact existing write scope. Validate its returned permission/repository scope and repeat provider-state preflight with that token. Then repeat the same workflow GET immediately before handoff. A mismatch at either later check refuses child handoff, but the management token may remain valid until expiry. 6. Only after both provider-state checks and the final workflow check, hand credentials to the exact reviewed controller child. No authentication-discovery route is supported. Repository restriction does **not** cryptographically restrict the organization-level self-hosted-runners permission to one runner group. The broker and reviewed controller enforce the selected canary policy, under the trusted-controller assumption. This boundary assumes a trusted controller UID and host administrators; file modes and a stdin pipe do not isolate either. The read-only preflight token is issued by the existing App; no separate user or organization credential is requested. The broker never adds Actions permission to the G02 App or mints the separate workflow authority. -A failed/ambiguous token request, failed policy check, expired credential, uncertain handoff or error leaves the intent and stops. There is no automatic retry, token refresh, token replacement or automatic revocation. Either issued token may remain valid until GitHub expires it even if no handoff occurs. A future operator approval must include both potential issuances, not just the eventual read/phase. The maximum remaining operation lifetime is ten minutes, approval expiry, or one minute before the management installation-token expiry, whichever occurs first. +A failed/ambiguous token request, failed policy check, expired credential, uncertain handoff or error leaves the intent and stops. This account-wide start is written before the POST can be dispatched, so a lost success response remains an unresolved lifecycle even when no token candidate is available to revoke. The v3 reusable-callee route makes one self-revocation attempt for each usable read-token candidate, including a token rejected for expiry or scope, and does not run provider preflight for rejected issuance. The per-attempt journal records `read_only_token_revocation_started` before DELETE. A caller-token collision records `read_only_token_revocation_skipped` and stops without a provider request, leaving the account intent pending. After a successful DELETE and a durable per-attempt `read_only_token_revoked` record, the broker syncs `read_token_lifecycle_confirmed` to the account ledger. If the per-attempt success record or the account confirmation cannot be synced, a best-effort `read_only_token_revocation_uncertain` record is attempted and the unmatched lifecycle intent blocks every later phase, including `inspect` and `cleanup`, before another authenticated request, management-token issuance or child handoff. An incomplete legacy reusable-callee claim without lifecycle or no-dispatch evidence also fails closed because its earlier POST outcome cannot be reconstructed. Correctly paired historical `read_token_revocation_started` / `read_token_revocation_confirmed` records remain accepted as completed history. The next open re-syncs the ledger before interpreting a visible confirmation. There is no automatic retry or token refresh/replacement. An uncertain revocation may leave the read token valid until expiry. A management token issued before a later failure may also remain valid until expiry; it is not automatically revoked. A future operator approval must include both potential issuances and the read-token revocation effect, not just the eventual read/phase. The maximum remaining operation lifetime is ten minutes, approval expiry, or one minute before the management installation-token expiry, whichever occurs first. + +The self-revocation request uses the [GitHub App installation-token revocation endpoint](https://docs.github.com/en/rest/apps/installations#revoke-an-installation-access-token), which accepts an installation access token and requires no additional permissions. This API contract is official documentation; fixture evidence does not prove live GitHub behavior for the selected installation. ## Historical discovery is an authentication effect diff --git a/docs/evidence/g01-live-canary.md b/docs/evidence/g01-live-canary.md index 6c53a0c8..e1e9cbff 100644 --- a/docs/evidence/g01-live-canary.md +++ b/docs/evidence/g01-live-canary.md @@ -409,22 +409,31 @@ live authorization unless explicitly excluded or separately disabled. maintainer-supplied existing organization App installation has `organization_self_hosted_runners: write`. No App permission, installation or repository setting was changed. Private App identifiers are omitted here. -- [ ] Verify that the separate secret broker keeps management credentials on the - controller. The organization-only App does not need repository access. Broker - and controller approvals must explicitly select the reviewed - [organization-only management scope](g01-organization-management-scope.md), - with separate workflow verification authority where required. Scope support - alone does not qualify issuance, provenance or execution. +- [x] Offline source/fixture evidence verifies the v3 broker builds the management + credential payload from authenticated issuance and sends it through the bounded + handoff to the selected controller child; the controller validates the signed + receipt/binding before reading credentials. The controller validates the + approved organization-only scope and keeps the workflow-verification token + distinct where a phase requires it. See the [broker evidence](g01-broker.md), + [organization-only scope ledger](g01-organization-management-scope.md), and + passing separate-process fixtures + TestControllerProcessHandoffActualBrokerSeparateProcess and + TestControllerProcessHandoffV3ActualBrokerChild. This closes source-level + custody evidence only. The current controller launches no worker child, so + actual worker credential inheritance, deployed identity, runtime isolation, + live issuance and production provenance remain unqualified. - [ ] Dedicated trusted controller and disposable worker environment is named. Prefer one Linux ARM64 worker on an explicitly chosen Docker endpoint/runtime; Docker Desktop presence alone is not authorization to use it. Native macOS requires a separately approved trusted execution identity. Do not share the management credential context with job code. -- [ ] Resource budget: **one active worker maximum**, one scale set, one session +- [x] Plan a maximum budget of **one active worker**, one scale set, one session owner, at most two small queued canary jobs per phase, 1 vCPU / 1 GiB worker - target after host headroom is checked, 10-minute job deadline. Increase any - limit only with new concrete authorization. No service-container workloads in - this protocol canary. + target, and a 10-minute job deadline; no service-container workloads. Raise + any limit only with new concrete authorization. +- [ ] Measure available host headroom and verify effective resource enforcement + on the selected trusted environment before execution; the budget above is a + target and does not establish current capacity. - [x] The offline adoption profile selects SDK `v0.5.0`/Go `1.27.3` and the byte-verified runner `2.338.0` ARM64 image. The archive, manifest and config digests are in the [runner checkpoint](g01-runner-release-checkpoint.md#2026-10-25-target-offer-archive-and-image-bytes); @@ -613,6 +622,82 @@ execution authorization. The current `gh auth status` scope list omits `read:user`; no scope was added. The GitHub App installation's selected-repository access remains unverified. +### Broker read-token revocation offline follow-up — 2026-11-03 + +On base `8537b8ba93ced1cea995dbd4d0402b98ef85897b`, local implementation +commit `47182b560b57ae6e9340e013c4a964a71ba6715c` adds a single revocation of the +v3 broker's narrowly scoped read-only installation token after its provider +preflight attempt. The broker durably records the revocation intent before +`DELETE /installation/token`; only an empty `204` response is confirmed. An +ambiguous result is journaled and stops before the final workflow check and +management-token mint. The G01 separate-process fixture now covers the read and +write mint scopes and verifies one read-token revocation. The broker journal +fixture also rejects persistence of the synthetic read token. + +The meaningful RED was +`go test -run '^TestBrokerExperimentV3StopsWhenReadOnlyTokenRevocationIsUncertain$' -count=1 .` +against the parent source. It failed because the synthetic revocation failure +still returned `controller_completed` after two mints and three workflow reads. +After the fix, the same case passes and the broker stops before the write +boundary. Go 1.27.3 results, with build/module caches and temporary build files +on the external worktree volume: + +| Check | Result | +|---|---| +| G02 `go test -run '^TestBrokerExperimentV3StopsWhenReadOnlyTokenRevocationIsUncertain$' -count=1 .` | PASS | +| G02 `go test -run '^TestBrokerExperimentV3' -count=1 .` | PASS (5.798s) | +| G02 v3 suite rerun after the journal redaction assertion | PASS (5.344s) | +| G02 `go test -count=1 -timeout=120s ./...` | PASS (all packages; library 48.110s) | +| G02 `go test -race -run '^TestBrokerExperimentV3(ReadOnlyPreflightRejectsBeforeWriteMint|StopsWhenReadOnlyTokenRevocationIsUncertain|FinalGETFencesCalleeBeforeMintAndHandoff)$' -count=1 -timeout=60s .` | PASS (17.057s) | +| G02 `go test -race -count=1 -timeout=120s ./...` | PASS (all packages; library 120.491s) | +| G02 `go vet ./...` | PASS | +| G01 `go test -tags=g01_pair_fixture -run '^(TestControllerProcessHandoffActualBrokerSeparateProcess|TestControllerProcessHandoffV3ActualBrokerChild)$' -count=1 -timeout=120s ./livecanary` | PASS (3.001s) | +| `git diff --check` | PASS | +| `git diff --unified=0 | rg -n 'BEGIN (RSA|OPENSSH|EC|PRIVATE)|ghs_[A-Za-z0-9_]{20,}|github_pat_[A-Za-z0-9_]{20,}|/Users/|/private/var|/tmp/'` | No matches (rg exit 1) | + +The initial G02 full-race attempt at a 45-second timeout expired while the +standalone compile test was running; the same full suite passed with a +120-second timeout. The first G01 process-fixture run exposed that its synthetic +server only modeled the previous single write-token flow; the fixture was +updated and both v1 and v3 process-handoff cases then passed. + +This is synthetic offline evidence only. No real GitHub token was minted or revoked, +and no provider, runner, Scale Set, Docker or host operation occurred. An +uncertain read-token revocation may still leave that token valid until expiry; +the later management token is not automatically revoked. The provider-read to +handoff race, trust-root/custody, runtime/isolation, conditional deletion and +separately authorized live canary gates remain open. Issue #1 and Project #2 +remain OPEN / In progress. + +### Read-token rejection and journal uncertainty regression — 2026-11-03 + +The independent Luna Max exact-head review of PR #159 at +`d8f387fdc7e367ac65e23a3b449ee7202aff2163` identified two P1 findings: rejected read-token scope/expiry +responses discarded a usable +token before cleanup, and a token equal to the caller verification credential +could reach provider-state preflight and DELETE. It also identified a P2 gap: +a successful DELETE followed by failed revoked-record sync did not append an +uncertainty marker. Local regression tests reproduced the P1s. The correction +retains only a lexically valid, bounded read-token candidate for one journaled +revocation attempt, never preflights rejected issuance, records/skips a +caller-token collision before provider-state preflight, and appends terminal +uncertainty after a failed revoked-result sync. Verification on the corrected +source commit `4f3d84f52402f21ec5b1031a19af92aaeb60b424`, using Go 1.27.3 +and Go caches on the external worktree volume: + +| Check | Result | +|---|---| +| RED on the pre-fix tree: rejected scope/expiry cleanup and caller-token collision regressions | FAIL as expected | +| G02 focused test `go test -run 'TestBroker(ReadOnlyRevocationRecordsUncertainWhenRevokedSyncFails|ExperimentV3RevokesTokenFromRejectedReadIssuance|ExperimentV3DoesNotRevokeVerificationTokenCollision)$'` | PASS (1.777s) | +| G02 `go test -count=1 -timeout=120s ./...` | PASS (all packages; library 47.211s) | +| G02 `go vet ./...` | PASS | +| G01 `go test -tags=g01_pair_fixture -run '^(TestControllerProcessHandoffActualBrokerSeparateProcess|TestControllerProcessHandoffV3ActualBrokerChild)$' -count=1 -timeout=120s ./livecanary` | PASS (3.773s) | + +No GitHub token, provider, runner, Scale Set or host operation was performed. +This offline correction does not change the selected SDK `v0.5.0` path or +close the still-unsupported conditional Scale Set deletion requirement; issue +#1 and Project #2 remain OPEN / In progress. + ## Minimal execution phases Run phases serially. Each phase begins by verifying that only its owned resources @@ -660,3 +745,87 @@ retaining uncertain resources for diagnosis and removing only verified disposabl resources. Existing manual runners remain the operational fallback. This plan does not implement production providers or adopt the upstream demo's forced busy-runner shutdown. + +### Account-wide read-token revocation uncertainty fence — 2026-11-03 + +The Luna Max review of PR #159 identified a compound journal failure: GitHub +DELETE succeeds, then both the per-attempt `read_only_token_revoked` record and +its uncertainty record fail to sync. The per-attempt durable intent stopped the +same invocation, but an approved later `inspect` could open a new attempt and +mint a management token. A regression run against base `1ea35587f5df5fa6` +reproduced the gap: the later inspect returned `controller_completed` after two +token mints. + +The correction adds a second intent/confirmation pair to the fixed account +admission ledger. It binds only phase plus attempt and journal device/inode, +contains no credential, and is synced before DELETE. Confirmation is written +only after the exact successful DELETE response and the per-attempt success +record are durable. Admission replay requires a unique start for the current +claim, no intervening event, and a matching confirmation. An unmatched start +refuses every later phase, including `inspect` and `cleanup`, before management +token issuance or child handoff. A visible confirmation is re-synced before the +next invocation may rely on it. The finite admission-ledger line bound now +includes this event pair for every allowed slot. + +Verification on the corrected local candidate, with Go 1.27.3 and the +gh-runnerd Go cache, module cache and temporary directory on the external +worktree volume: + +| Check | Result | +|---|---| +| RED at base `1ea3558`: double per-attempt sync failure followed by inspect | FAIL as expected; inspect completed after two token mints | +| Focused revocation, confirmed-history replay, inspect-fence and ledger-bound tests | PASS | +| G02 `go test -count=1 -timeout=120s ./...` | PASS (all packages; library 46.635s) | +| G02 `go vet ./...` | PASS | +| G02 focused `go test -race` for compound failure, confirmed-history replay and revocation-sync failure | PASS (4.635s) | +| `git diff --check` | PASS | + +These are synthetic offline results; no real GitHub token was minted or +revoked, and no provider, runner, Scale Set, Docker or host operation occurred. +This closes the identified offline admission-fence gap. The read-to-handoff +race, management-token expiry/revocation, conditional Scale Set deletion, +trusted runtime and custody, production provenance, live readiness, and +separately authorized canary remain unresolved. Issue #1 and Project #2 remain +OPEN / In progress. + +### Read-token issuance response-loss fence — 2026-11-03 + +The independent Luna Max review of PR #159 at `a35220b089d3a3bd79ab1575620892f138739a42` +identified a separate P1: the account admission ledger did not record the +read-token lifecycle until after a usable token response existed. If GitHub +accepted the POST but its response was lost, no token candidate was available +for DELETE and a later `inspect` could mint management authority over an +unresolved read-token issuance. The synthetic regression reproduced that +sequence: no account-level intent was present and the later inspect completed. + +The local correction syncs `read_token_lifecycle_started` immediately before +the read-token HTTP request is dispatched. A lost response therefore leaves a +durable account-wide barrier. If a v3 attempt ends before the dispatch boundary, +it records `read_token_request_not_dispatched`, allowing safe later recovery. +An incomplete historical reusable-callee claim with neither lifecycle nor +no-dispatch evidence now fails closed. Correctly paired historical +`read_token_revocation_started` / `read_token_revocation_confirmed` records +remain valid. The finding has a causal regression for accepted-POST/response +loss, sync failure before request dispatch, no-dispatch recovery, legacy +incomplete claims and both current and historical confirmed-pair replay. + +Local verification on the candidate based on `a35220b` used Go 1.27.3 and +external-volume Go caches: + +| Check | Result | +|---|---| +| RED at `a35220b`: accepted read-token POST with lost response, then inspect | FAIL as expected; no account intent, inspect completed and minted again | +| Focused lifecycle, failure-boundary, replay and recovery tests | PASS (3.113s) | +| G02 `go test -race -count=1 -timeout=210s ./...` | PASS (all packages; library 129.994s) | +| G02 `go vet ./...` | PASS | +| `git diff --check` | PASS | + +The source and docs are still a local candidate; fresh independent and +GitHub Codex review of its exact head and hosted PR quick checks remain pending. +GitHub issue, PR and Project metadata were read through `gh`; no provider +credential/token, runner, Scale Set, Docker or host operation was used. The +selected SDK path is unchanged. Production provenance and trust +roots, broker-only execution, management-token lifecycle, conditional Scale Set +deletion, trusted runtime/isolation/resource qualification, the authorized +Linux ARM64/JIT canary and sanitized live exit evidence remain open. Issue #1 +stays OPEN and Project #2 stays In progress. diff --git a/experiments/g01-scaleset/livecanary/controller_process_broker_fixture_test.go b/experiments/g01-scaleset/livecanary/controller_process_broker_fixture_test.go index 59dfde45..3970fa46 100644 --- a/experiments/g01-scaleset/livecanary/controller_process_broker_fixture_test.go +++ b/experiments/g01-scaleset/livecanary/controller_process_broker_fixture_test.go @@ -26,6 +26,7 @@ type processBrokerHTTP struct { attempt string calls []string mints int + revocations int failPreflight bool groupFault bool } @@ -68,20 +69,41 @@ func (f *processBrokerHTTP) RoundTrip(r *http.Request) (*http.Response, error) { f.t.Error("mint preceded durable intent") } body, _ := io.ReadAll(r.Body) + var request struct { + RepositoryIDs []int64 `json:"repository_ids"` + Permissions map[string]string `json:"permissions"` + } + if json.Unmarshal(body, &request) != nil || (request.Permissions["organization_self_hosted_runners"] != "read" && request.Permissions["organization_self_hosted_runners"] != "write") { + f.t.Error("mint permission set was invalid") + } organizationOnly := f.approval != nil && f.approval.ManagementScope == "organization-only" - if !organizationOnly && !bytes.Contains(body, []byte(`"repository_ids":[42]`)) { + if !organizationOnly && !reflect.DeepEqual(request.RepositoryIDs, []int64{42}) { f.t.Error("mint not scoped") } status = 201 - payload = map[string]any{"token": "synthetic-process-installation-token", "expires_at": time.Now().Add(time.Hour), "permissions": map[string]string{"organization_self_hosted_runners": "write", "metadata": "read"}, "repository_selection": "selected", "repositories": []any{repo}} + token := "synthetic-process-installation-token" + if request.Permissions["organization_self_hosted_runners"] == "read" { + token = "synthetic-process-read-installation-token" + } + payload = map[string]any{"token": token, "expires_at": time.Now().Add(time.Hour), "permissions": request.Permissions, "repository_selection": "selected", "repositories": []any{repo}} if organizationOnly { - var request map[string]any - _ = json.Unmarshal(body, &request) - if len(request) != 1 { + var organizationRequest map[string]any + _ = json.Unmarshal(body, &organizationRequest) + if len(organizationRequest) != 1 { f.t.Error("organization-only mint widened") } - payload = map[string]any{"token": "synthetic-process-installation-token", "expires_at": time.Now().Add(time.Hour), "permissions": map[string]string{"organization_self_hosted_runners": "write"}} + payload = map[string]any{"token": token, "expires_at": time.Now().Add(time.Hour), "permissions": request.Permissions} + } + case "/installation/token": + f.revocations++ + if r.Method != http.MethodDelete || r.Header.Get("Authorization") != "Bearer synthetic-process-read-installation-token" { + f.t.Error("read-token revocation used the wrong method or authority") } + stored, e := os.ReadFile(filepath.Join(f.attempt, "broker.jsonl")) + if e != nil || !bytes.Contains(stored, []byte("read_only_token_revocation_started")) { + f.t.Error("read-token revocation preceded durable intent") + } + status = http.StatusNoContent case "/installation/repositories": payload = map[string]any{"total_count": 1, "repositories": []any{repo}} case "/repos/fixture-org/canary": @@ -101,7 +123,10 @@ func (f *processBrokerHTTP) RoundTrip(r *http.Request) (*http.Response, error) { default: return nil, fmt.Errorf("unexpected synthetic route") } - raw, _ := json.Marshal(payload) + var raw []byte + if status != http.StatusNoContent { + raw, _ = json.Marshal(payload) + } return &http.Response{StatusCode: status, Header: make(http.Header), Body: io.NopCloser(bytes.NewReader(raw)), ContentLength: int64(len(raw))}, nil } func TestControllerProcessHandoffActualBrokerSeparateProcess(t *testing.T) { diff --git a/experiments/g01-scaleset/livecanary/controller_v3_process_fixture_test.go b/experiments/g01-scaleset/livecanary/controller_v3_process_fixture_test.go index 26d5058b..9ad212fb 100644 --- a/experiments/g01-scaleset/livecanary/controller_v3_process_fixture_test.go +++ b/experiments/g01-scaleset/livecanary/controller_v3_process_fixture_test.go @@ -279,12 +279,12 @@ func TestControllerProcessHandoffV3ActualBrokerChild(t *testing.T) { if tc.wantSuccess { receipt, receiptErr := handoff.DecodeStrictReceipt(observedReceipt) binding, bindingErr := handoff.DecodeStrictBinding(observedBinding) - if err != nil || result.Status != "controller_completed" || effects.WorkflowSignatures != 1 || effects.BindingSignatures != 1 || fixture.transport.base.mints != 1 || receiptErr != nil || bindingErr != nil || receipt.Callee != fixture.controller.request.Callee || binding.Request != fixture.controller.request || controllerEffects["factory_calls"] != 1 || controllerEffects["create_calls"] != 1 || len(events) != 7 { + if err != nil || result.Status != "controller_completed" || effects.WorkflowSignatures != 1 || effects.BindingSignatures != 1 || fixture.transport.base.mints != 2 || fixture.transport.base.revocations != 1 || receiptErr != nil || bindingErr != nil || receipt.Callee != fixture.controller.request.Callee || binding.Request != fixture.controller.request || controllerEffects["factory_calls"] != 1 || controllerEffects["create_calls"] != 1 || len(events) != 7 { t.Fatalf("positive child handoff: result=%+v err=%v effects=%+v mints=%d calls=%v controller_effects=%v journal_events=%d receipt_err=%v binding_err=%v", result, err, effects, fixture.transport.base.mints, fixture.transport.base.calls, controllerEffects, len(events), receiptErr, bindingErr) } return } - if err == nil || result.Status == "controller_completed" || effects.WorkflowSignatures != 1 || effects.BindingSignatures != 1 || fixture.transport.base.mints != 1 || controllerEffects["factory_calls"] != 0 || controllerEffects["create_calls"] != 0 || len(events) != 0 { + if err == nil || result.Status == "controller_completed" || effects.WorkflowSignatures != 1 || effects.BindingSignatures != 1 || fixture.transport.base.mints != 2 || fixture.transport.base.revocations != 1 || controllerEffects["factory_calls"] != 0 || controllerEffects["create_calls"] != 0 || len(events) != 0 { t.Fatalf("rejected child handoff crossed effect boundary: result=%+v err=%v effects=%+v mints=%d calls=%v controller_effects=%v journal_events=%d", result, err, effects, fixture.transport.base.mints, fixture.transport.base.calls, controllerEffects, len(events)) } }) diff --git a/experiments/g02-auth/broker.go b/experiments/g02-auth/broker.go index 6ada3280..2d3dd555 100644 --- a/experiments/g02-auth/broker.go +++ b/experiments/g02-auth/broker.go @@ -62,10 +62,10 @@ const organizationOnlyManagementScope = "organization-only" func brokerSlotAllowed(slot string) bool { return brokerPhases[slot] || brokerSpecialSlots[slot] } -// One header, one claim and one completion for every finite schema slot, and -// the final empty split element from the required trailing newline. This is a -// structural bound derived from the actual slot schema, not an unbounded log. -func brokerLedgerMaxLines() int { return 1 + 2*(len(brokerPhases)+len(brokerSpecialSlots)) + 1 } +// One header, one claim and one completion for every finite schema slot, up to +// one read-token revocation start/confirmation pair per slot, and the final +// empty split element from the required trailing newline. +func brokerLedgerMaxLines() int { return 1 + 4*(len(brokerPhases)+len(brokerSpecialSlots)) + 1 } const ( // Terminal collection has seven five-second cadence gaps in production. @@ -149,6 +149,36 @@ func brokerPairedAuthorityDeadline(parent context.Context, a BrokerApproval, pla return deadline, nil } +func revokeBrokerReadOnlyToken(ctx context.Context, api *brokerAPI, j *brokerJournal, claim *brokerAdmission, token, verificationToken string) error { + if !validBrokerToken(token) || len(token) > 1024 || claim == nil || !claim.readTokenLifecyclePending || claim.check() != nil { + return errBroker + } + if token == verificationToken { + if j.append("read_only_token_revocation_skipped", map[string]any{"reason": "verification_token_collision"}) != nil { + return errBroker + } + return errBroker + } + if j.append("read_only_token_revocation_started", nil) != nil { + return errBroker + } + if api.revokeInstallationToken(ctx, token) != nil { + _ = j.append("read_only_token_revocation_uncertain", map[string]any{"reason": "revoke_request_failed_or_ambiguous"}) + return errBroker + } + if j.append("read_only_token_revoked", nil) != nil { + // The revoked record may have reached disk without a durable sync. A + // later uncertainty record is terminal and overrides that observation. + _ = j.append("read_only_token_revocation_uncertain", map[string]any{"reason": "revocation_result_persistence_failed"}) + return errBroker + } + if claim.confirmReadTokenLifecycle() != nil { + _ = j.append("read_only_token_revocation_uncertain", map[string]any{"reason": "admission_lifecycle_confirmation_failed"}) + return errBroker + } + return nil +} + func brokerExecute(parent context.Context, a BrokerApproval, input brokerInput, path string, api *brokerAPI, plan *brokerControllerPlan) (BrokerResult, error) { return brokerExecuteWithProvenance(parent, a, input, path, api, plan, nil) } @@ -192,7 +222,7 @@ func brokerExecuteWithProvenance(parent context.Context, a BrokerApproval, input j = claimed[0] } else { var err error - j, err = openBrokerJournal(path, a) + j, err = openBrokerJournalForExecution(parent, path, a) if err != nil { return BrokerResult{}, errBroker } @@ -237,6 +267,14 @@ func brokerExecuteWithProvenance(parent context.Context, a BrokerApproval, input return BrokerResult{}, errBroker } defer claim.close() + readTokenLifecycleStarted := false + if v3ReusableCallee { + defer func() { + if !readTokenLifecycleStarted { + _ = claim.recordReadTokenRequestNotDispatched() + } + }() + } guard := func(checkPrepared bool) error { if claim.check() != nil || j.check() != nil { return errBroker @@ -298,17 +336,32 @@ func brokerExecuteWithProvenance(parent context.Context, a BrokerApproval, input if j.append("token_request_started", nil) != nil { return BrokerResult{}, errBroker } - readOnly, err := api.mintReadOnly(ctx, a, cred) + readOnly, err := api.mintReadOnlyWithIntent(ctx, a, cred, func() error { + if err := claim.beginReadTokenLifecycle(); err != nil { + return err + } + readTokenLifecycleStarted = true + return nil + }) if err != nil { + if readOnly.Token != "" { + _ = revokeBrokerReadOnlyToken(ctx, api, j, claim, readOnly.Token, input.VerificationToken) + } + return BrokerResult{}, errBroker + } + if readOnly.Token != "" && readOnly.Token == input.VerificationToken { + _ = revokeBrokerReadOnlyToken(ctx, api, j, claim, readOnly.Token, input.VerificationToken) return BrokerResult{}, errBroker } readCtx, stopRead := context.WithDeadline(ctx, readOnly.ExpiresAt.Add(-time.Minute)) preflightErr := api.preflight(readCtx, a, readOnly.Token) stopRead() - if input.VerificationToken == readOnly.Token || preflightErr != nil { - return BrokerResult{}, errBroker + readOnlyVerified := preflightErr == nil + verifiedJournaled := true + if readOnlyVerified { + verifiedJournaled = j.append("read_only_token_verified", map[string]any{"expires_at": readOnly.ExpiresAt, "app_id": identity.ID, "installation_id": install.ID}) == nil } - if j.append("read_only_token_verified", map[string]any{"expires_at": readOnly.ExpiresAt, "app_id": identity.ID, "installation_id": install.ID}) != nil { + if revokeBrokerReadOnlyToken(ctx, api, j, claim, readOnly.Token, input.VerificationToken) != nil || !readOnlyVerified || !verifiedJournaled { return BrokerResult{}, errBroker } // The final workflow GET is deliberately after read-only provider diff --git a/experiments/g02-auth/broker_admission.go b/experiments/g02-auth/broker_admission.go index 84c0f098..22174c54 100644 --- a/experiments/g02-auth/broker_admission.go +++ b/experiments/g02-auth/broker_admission.go @@ -12,6 +12,8 @@ import ( "strconv" "syscall" "time" + + "github.com/1XP-AI/gh-runnerd/experiments/g02-auth/handoff" ) // One finite experiment per native controller account. Never selected by HOME, @@ -75,9 +77,10 @@ func brokerOwnedDirectory(i os.FileInfo, private bool) bool { return ok && int(s.Uid) == os.Geteuid() } -// The finite slot schema has eight controller phases plus discovery and paired -// terminal. Each stored controller authority is bounded by the 16 KiB input -// contract; allow one claim and completion for every schema slot. +// The finite slot schema is the union of the named phases and special slots. +// Each stored controller authority is bounded by the 16 KiB input +// contract; allow one claim/completion and one read-token revocation +// start/confirmation pair for every schema slot. const maxBrokerLedgerBytes = 512 << 10 type brokerInode struct { @@ -146,6 +149,12 @@ type brokerClaimEvent struct { Snapshot brokerInode `json:"snapshot"` SnapshotDigest string `json:"snapshot_digest,omitempty"` } +type brokerReadTokenLifecycleEvent struct { + Kind string `json:"kind"` + Slot string `json:"slot"` + Attempt brokerInode `json:"attempt"` + Journal brokerInode `json:"journal"` +} type brokerLedgerHeader struct { Version int `json:"version"` Resource string `json:"resource"` @@ -161,6 +170,7 @@ type brokerAdmission struct { rootInfo, parentInfo, fileInfo os.FileInfo data []byte event brokerClaimEvent + readTokenLifecyclePending bool sync func(*os.Root) error } @@ -217,7 +227,7 @@ func (c *brokerAdmission) append(value any) error { return nil } func (c *brokerAdmission) complete() error { - if c.check() != nil { + if c == nil || c.readTokenLifecyclePending || c.check() != nil { return errBroker } e := c.event @@ -227,6 +237,38 @@ func (c *brokerAdmission) complete() error { } return c.check() } +func (c *brokerAdmission) beginReadTokenLifecycle() error { + if c == nil || c.readTokenLifecyclePending || c.check() != nil { + return errBroker + } + event := brokerReadTokenLifecycleEvent{Kind: "read_token_lifecycle_started", Slot: c.event.Slot, Attempt: c.event.Attempt, Journal: c.event.Journal} + if c.append(event) != nil || c.check() != nil { + return errBroker + } + c.readTokenLifecyclePending = true + return nil +} +func (c *brokerAdmission) confirmReadTokenLifecycle() error { + if c == nil || !c.readTokenLifecyclePending || c.check() != nil { + return errBroker + } + event := brokerReadTokenLifecycleEvent{Kind: "read_token_lifecycle_confirmed", Slot: c.event.Slot, Attempt: c.event.Attempt, Journal: c.event.Journal} + if c.append(event) != nil || c.check() != nil { + return errBroker + } + c.readTokenLifecyclePending = false + return nil +} +func (c *brokerAdmission) recordReadTokenRequestNotDispatched() error { + if c == nil || c.readTokenLifecyclePending || c.event.Provenance == nil || c.event.Provenance.Source != handoff.ReusableCalleeSource || c.check() != nil { + return errBroker + } + event := brokerReadTokenLifecycleEvent{Kind: "read_token_request_not_dispatched", Slot: c.event.Slot, Attempt: c.event.Attempt, Journal: c.event.Journal} + if c.append(event) != nil || c.check() != nil { + return errBroker + } + return nil +} func openBrokerAdmission(directory string, a BrokerApproval, j *brokerJournal, p *brokerControllerPlan, syncRoot func(*os.Root) error) (claim *brokerAdmission, err error) { if syncRoot == nil || !filepath.IsAbs(directory) || filepath.Clean(directory) != directory { return nil, errBroker @@ -312,61 +354,121 @@ func openBrokerAdmission(directory string, a BrokerApproval, j *brokerJournal, p var workerBinding *brokerWorkerBinding var authority *brokerControllerAuthority provenanceNonces := map[string]bool{} + readTokenLifecycleSlots := map[string]bool{} + readTokenRequestNotDispatchedSlots := map[string]bool{} latestSlot := "" + var pendingReadTokenLifecycle *brokerReadTokenLifecycleEvent for _, line := range lines[1 : len(lines)-1] { - var event brokerClaimEvent - if decodeBrokerJSON(line, &event, true) != nil || !validBrokerClaimEvent(a, event) { + var kind struct { + Kind string `json:"kind"` + } + if decodeBrokerJSON(line, &kind, false) != nil { return nil, errBroker } - switch event.Kind { - case "claim": - if event.Provenance != nil { - nonce := event.Provenance.ReceiptNonce - if provenanceNonces[nonce] { + switch kind.Kind { + case "read_token_lifecycle_started", "read_token_lifecycle_confirmed", "read_token_revocation_started", "read_token_revocation_confirmed", "read_token_request_not_dispatched": + var event brokerReadTokenLifecycleEvent + if decodeBrokerJSON(line, &event, true) != nil || !validBrokerReadTokenLifecycleEvent(event) { + return nil, errBroker + } + switch event.Kind { + case "read_token_lifecycle_started", "read_token_revocation_started": + old, ok := slots[event.Slot] + if pendingReadTokenLifecycle != nil || readTokenLifecycleSlots[event.Slot] || latestSlot != event.Slot || !ok || done[event.Slot] || old.Attempt != event.Attempt || old.Journal != event.Journal { return nil, errBroker } - provenanceNonces[nonce] = true - } - for previous, receipt := range slots { - if (!done[previous] && event.Slot != "inspect" && event.Slot != "cleanup") || receipt.Attempt == event.Attempt || receipt.Journal == event.Journal || (event.Controller != nil && receipt.Controller != nil && receipt.Snapshot == event.Snapshot) || (event.Worker != nil && receipt.Worker != nil && receipt.Worker.State == event.Worker.State) { + readTokenLifecycleSlots[event.Slot] = true + pendingReadTokenLifecycle = &event + case "read_token_lifecycle_confirmed", "read_token_revocation_confirmed": + if pendingReadTokenLifecycle == nil || !sameBrokerReadTokenLifecycle(*pendingReadTokenLifecycle, event) { + return nil, errBroker + } + pendingReadTokenLifecycle = nil + case "read_token_request_not_dispatched": + old, ok := slots[event.Slot] + if latestSlot != event.Slot || !ok || done[event.Slot] || readTokenRequestNotDispatchedSlots[event.Slot] || old.Attempt != event.Attempt || old.Journal != event.Journal || old.Provenance == nil || old.Provenance.Source != handoff.ReusableCalleeSource { return nil, errBroker } + if pendingReadTokenLifecycle != nil { + if pendingReadTokenLifecycle.Kind != "read_token_lifecycle_started" || !sameBrokerReadTokenIdentity(*pendingReadTokenLifecycle, event) { + return nil, errBroker + } + pendingReadTokenLifecycle = nil + } else if readTokenLifecycleSlots[event.Slot] { + return nil, errBroker + } + readTokenLifecycleSlots[event.Slot] = true + readTokenRequestNotDispatchedSlots[event.Slot] = true } - latestSlot = event.Slot - if _, ok := slots[event.Slot]; ok { + case "claim", "complete": + if pendingReadTokenLifecycle != nil { return nil, errBroker } - if event.Controller != nil { - if binding != nil && *binding != *event.Controller { - return nil, errBroker + var event brokerClaimEvent + if decodeBrokerJSON(line, &event, true) != nil || !validBrokerClaimEvent(a, event) { + return nil, errBroker + } + switch event.Kind { + case "claim": + if event.Provenance != nil { + nonce := event.Provenance.ReceiptNonce + if provenanceNonces[nonce] { + return nil, errBroker + } + provenanceNonces[nonce] = true } - if event.Authority == nil { - return nil, errBroker + for previous, receipt := range slots { + if (!done[previous] && event.Slot != "inspect" && event.Slot != "cleanup") || receipt.Attempt == event.Attempt || receipt.Journal == event.Journal || (event.Controller != nil && receipt.Controller != nil && receipt.Snapshot == event.Snapshot) || (event.Worker != nil && receipt.Worker != nil && receipt.Worker.State == event.Worker.State) { + return nil, errBroker + } } - if !brokerAuthorityTransition(authority, event.Authority) { + latestSlot = event.Slot + if _, ok := slots[event.Slot]; ok { return nil, errBroker } - binding = event.Controller - if event.Worker != nil { - if workerBinding != nil && *workerBinding != *event.Worker { + if event.Controller != nil { + if binding != nil && *binding != *event.Controller { + return nil, errBroker + } + if event.Authority == nil { return nil, errBroker } - workerBinding = event.Worker + if !brokerAuthorityTransition(authority, event.Authority) { + return nil, errBroker + } + binding = event.Controller + if event.Worker != nil { + if workerBinding != nil && *workerBinding != *event.Worker { + return nil, errBroker + } + workerBinding = event.Worker + } + authority = event.Authority } - authority = event.Authority - } - slots[event.Slot] = event - case "complete": - old, ok := slots[event.Slot] - event.Kind = "claim" - if !ok || latestSlot != event.Slot || done[event.Slot] || brokerDigest(old) != brokerDigest(event) { - return nil, errBroker + slots[event.Slot] = event + case "complete": + old, ok := slots[event.Slot] + event.Kind = "claim" + if !ok || latestSlot != event.Slot || done[event.Slot] || readTokenRequestNotDispatchedSlots[event.Slot] || brokerDigest(old) != brokerDigest(event) { + return nil, errBroker + } + done[event.Slot] = true } - done[event.Slot] = true default: return nil, errBroker } } + if pendingReadTokenLifecycle != nil { + return nil, errBroker + } + for slot, event := range slots { + if !done[slot] && event.Provenance != nil && event.Provenance.Source == handoff.ReusableCalleeSource && !readTokenLifecycleSlots[slot] { + // Older reusable-callee claims may predate account-wide token + // lifecycle intents. Their incomplete state cannot prove whether a + // read-token POST was accepted, so recovery must fail closed. + return nil, errBroker + } + } slot := a.Phase if a.Mode == "discover-actions-host" { slot = a.Mode @@ -499,3 +601,19 @@ func validBrokerClaimEvent(a BrokerApproval, e brokerClaimEvent) bool { c.Phases = nil return e.Controller.Ownership == brokerDigest(c) } + +func validBrokerReadTokenLifecycleEvent(e brokerReadTokenLifecycleEvent) bool { + return (e.Kind == "read_token_lifecycle_started" || e.Kind == "read_token_lifecycle_confirmed" || e.Kind == "read_token_revocation_started" || e.Kind == "read_token_revocation_confirmed" || e.Kind == "read_token_request_not_dispatched") && + brokerSlotAllowed(e.Slot) && e.Slot != "discover-actions-host" && + e.Attempt.Device != 0 && e.Attempt.Inode != 0 && e.Journal.Device != 0 && e.Journal.Inode != 0 +} + +func sameBrokerReadTokenIdentity(started, event brokerReadTokenLifecycleEvent) bool { + return started.Slot == event.Slot && started.Attempt == event.Attempt && started.Journal == event.Journal +} + +func sameBrokerReadTokenLifecycle(started, confirmed brokerReadTokenLifecycleEvent) bool { + legacyPair := started.Kind == "read_token_revocation_started" && confirmed.Kind == "read_token_revocation_confirmed" + lifecyclePair := started.Kind == "read_token_lifecycle_started" && confirmed.Kind == "read_token_lifecycle_confirmed" + return (legacyPair || lifecyclePair) && sameBrokerReadTokenIdentity(started, confirmed) +} diff --git a/experiments/g02-auth/broker_admission_test.go b/experiments/g02-auth/broker_admission_test.go index 05ff2bb7..8ee53a5c 100644 --- a/experiments/g02-auth/broker_admission_test.go +++ b/experiments/g02-auth/broker_admission_test.go @@ -109,8 +109,8 @@ func TestBrokerFinitePhasesAndUnknownRetention(t *testing.T) { } func TestBrokerLedgerCapacityDerivesFromFiniteSlotSchema(t *testing.T) { - want := 1 + 2*(len(brokerPhases)+len(brokerSpecialSlots)) + 1 - if got := brokerLedgerMaxLines(); got != want || got != 28 || len(brokerPhases) != 11 || len(brokerSpecialSlots) != 2 { + want := 1 + 4*(len(brokerPhases)+len(brokerSpecialSlots)) + 1 + if got := brokerLedgerMaxLines(); got != want || got != 54 || len(brokerPhases) != 11 || len(brokerSpecialSlots) != 2 { t.Fatalf("ledger line bound=%d want schema-derived %d", got, want) } if brokerSlotAllowed("unreviewed-slot") || !brokerSlotAllowed("paired-terminal") || !brokerSlotAllowed("discover-actions-host") || !brokerSlotAllowed("session-replacement") || !brokerSlotAllowed("before-jit") { @@ -118,6 +118,58 @@ func TestBrokerLedgerCapacityDerivesFromFiniteSlotSchema(t *testing.T) { } } +func TestBrokerAdmissionAllowsLaterInspectAfterConfirmedReadTokenLifecycle(t *testing.T) { + for _, tc := range []struct { + name string + started string + confirmed string + }{ + {name: "legacy revocation pair", started: "read_token_revocation_started", confirmed: "read_token_revocation_confirmed"}, + {name: "current lifecycle pair", started: "read_token_lifecycle_started", confirmed: "read_token_lifecycle_confirmed"}, + } { + t.Run(tc.name, func(t *testing.T) { + a, credential, api, fixture, attempt := newBrokerFixture(t) + a.Mode, a.Phase = "controller", "create" + parent := filepath.Dir(attempt) + plan := brokerTestPlan(t, &a, parent, func(context.Context, []byte, string) error { return nil }) + if _, err := brokerExecute(context.Background(), a, brokerInput{PEM: string(credential.PEM)}, attempt, api, plan); err != nil { + t.Fatalf("complete create phase: %v", err) + } + ledgerPath := filepath.Join(fixture.admissionRoot, "broker-admission.jsonl") + ledger, err := os.ReadFile(ledgerPath) + if err != nil { + t.Fatalf("read admission ledger: %v", err) + } + records := strings.Split(strings.TrimSuffix(string(ledger), "\n"), "\n") + if len(records) != 3 { + t.Fatalf("unexpected admission fixture records: %d", len(records)) + } + var createClaim brokerClaimEvent + if decodeBrokerJSON([]byte(records[1]), &createClaim, true) != nil || createClaim.Kind != "claim" { + t.Fatal("decode create admission claim") + } + started, _ := json.Marshal(brokerReadTokenLifecycleEvent{Kind: tc.started, Slot: createClaim.Slot, Attempt: createClaim.Attempt, Journal: createClaim.Journal}) + confirmed, _ := json.Marshal(brokerReadTokenLifecycleEvent{Kind: tc.confirmed, Slot: createClaim.Slot, Attempt: createClaim.Attempt, Journal: createClaim.Journal}) + replayed := strings.Join([]string{records[0], records[1], string(started), string(confirmed), records[2], ""}, "\n") + if err := os.WriteFile(ledgerPath, []byte(replayed), 0600); err != nil { + t.Fatalf("write synthetic confirmed token history: %v", err) + } + + a.Phase = "inspect" + a.ExpiresAt = a.ExpiresAt.Add(time.Minute) + fixture.root = filepath.Join(parent, "inspect") + inspectPlan := brokerTestPlan(t, &a, parent, func(context.Context, []byte, string) error { return nil }) + inspectPlan.controller.Phases = []string{"inspect", "cleanup"} + inspectPlan.raw, _ = json.Marshal(inspectPlan.controller) + a.ControllerApprovalSHA256 = brokerBytesDigest(inspectPlan.raw) + inspectPlan.approval = a + if _, err := brokerExecute(context.Background(), a, brokerInput{PEM: string(credential.PEM)}, fixture.root, api, inspectPlan); err != nil || fixture.tokenCalls != 2 { + t.Fatalf("confirmed read-token history blocked later inspect: err=%v mints=%d calls=%v", err, fixture.tokenCalls, fixture.calls) + } + }) + } +} + func TestBrokerAdmissionRejectsDuplicateProvenanceNonceBeforeAppend(t *testing.T) { a, c, api, f, root := newBrokerFixture(t) parent := filepath.Dir(root) diff --git a/experiments/g02-auth/broker_api.go b/experiments/g02-auth/broker_api.go index 2af04b51..fa496a50 100644 --- a/experiments/g02-auth/broker_api.go +++ b/experiments/g02-auth/broker_api.go @@ -52,7 +52,7 @@ func (t brokerTransport) RoundTrip(r *http.Request) (*http.Response, error) { return nil, errBroker } // Apply duplicate-key/depth checks to identity responses reused from G02 too. - if res.StatusCode >= 200 && res.StatusCode < 300 { + if res.StatusCode >= 200 && res.StatusCode < 300 && !(res.StatusCode == http.StatusNoContent && len(data) == 0) { var value any if decodeBrokerJSON(data, &value, false) != nil { return nil, errBroker @@ -96,6 +96,10 @@ func newBrokerHTTPClient(fixture http.RoundTripper) *http.Client { return &http.Client{Transport: transport, Timeout: 10 * time.Second, CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }} } func (a *brokerAPI) call(ctx context.Context, method, path, authorization string, body any, status int, out any) error { + return a.callWithBeforeRequest(ctx, method, path, authorization, body, status, out, nil) +} + +func (a *brokerAPI) callWithBeforeRequest(ctx context.Context, method, path, authorization string, body any, status int, out any, beforeRequest func() error) error { var data []byte var err error if body != nil { @@ -114,6 +118,9 @@ func (a *brokerAPI) call(ctx context.Context, method, path, authorization string r.Header.Set("Content-Type", "application/json") r.Header.Set("Accept-Encoding", "identity") r.Header.Set("User-Agent", "gh-runnerd-g01-broker-evidence") + if beforeRequest != nil && beforeRequest() != nil { + return errBroker + } res, err := a.client.Do(r) if err != nil { return errBroker @@ -123,7 +130,16 @@ func (a *brokerAPI) call(ctx context.Context, method, path, authorization string return errBroker } data, err = io.ReadAll(io.LimitReader(res.Body, maxResponseBytes+1)) - if err != nil || len(data) > maxResponseBytes || res.StatusCode != status || decodeBrokerJSON(data, out, false) != nil { + if err != nil || len(data) > maxResponseBytes || res.StatusCode != status { + return errBroker + } + if out == nil { + if len(data) != 0 { + return errBroker + } + return nil + } + if decodeBrokerJSON(data, out, false) != nil { return errBroker } return nil @@ -179,17 +195,24 @@ func (r brokerIssued) matchesReadOnlyScope(a BrokerApproval) bool { } func (a *brokerAPI) mint(ctx context.Context, approval BrokerApproval, cred Credential) (brokerIssued, error) { - return a.mintWithPermissions(ctx, approval, cred, false) + return a.mintWithPermissions(ctx, approval, cred, false, nil) } -func (a *brokerAPI) mintReadOnly(ctx context.Context, approval BrokerApproval, cred Credential) (brokerIssued, error) { - if approval.GroupPolicy.Version != 2 || !approval.GroupPolicy.Valid(approval.Organization+"/"+approval.Repository, "") || approval.GroupPolicy.OrganizationID != approval.OrganizationID { +func (a *brokerAPI) mintReadOnlyWithIntent(ctx context.Context, approval BrokerApproval, cred Credential, beforeRequest func() error) (brokerIssued, error) { + if beforeRequest == nil || approval.GroupPolicy.Version != 2 || !approval.GroupPolicy.Valid(approval.Organization+"/"+approval.Repository, "") || approval.GroupPolicy.OrganizationID != approval.OrganizationID { return brokerIssued{}, errBroker } - return a.mintWithPermissions(ctx, approval, cred, true) + return a.mintWithPermissions(ctx, approval, cred, true, beforeRequest) +} + +func (a *brokerAPI) revokeInstallationToken(ctx context.Context, token string) error { + if !validBrokerToken(token) || len(token) > 1024 { + return errBroker + } + return a.call(ctx, http.MethodDelete, "/installation/token", "Bearer "+token, nil, http.StatusNoContent, nil) } -func (a *brokerAPI) mintWithPermissions(ctx context.Context, approval BrokerApproval, cred Credential, readOnly bool) (brokerIssued, error) { +func (a *brokerAPI) mintWithPermissions(ctx context.Context, approval BrokerApproval, cred Credential, readOnly bool, beforeRequest func() error) (brokerIssued, error) { if !approval.GroupPolicy.Valid(approval.Organization+"/"+approval.Repository, "") || (approval.GroupPolicy.Version == 2 && approval.GroupPolicy.OrganizationID != approval.OrganizationID) || (approval.ManagementScope != "" && approval.ManagementScope != organizationOnlyManagementScope) { return brokerIssued{}, errBroker } @@ -212,14 +235,35 @@ func (a *brokerAPI) mintWithPermissions(ctx context.Context, approval BrokerAppr Permissions map[string]string `json:"permissions"` }{repositories, permissions} var result brokerIssued - if a.call(ctx, "POST", "/app/installations/"+strconv.FormatInt(approval.InstallationID, 10)+"/access_tokens", "Bearer "+jwt, body, 201, &result) != nil || !validBrokerToken(result.Token) || len(result.Token) > 1024 || !result.ExpiresAt.After(a.now().Add(time.Minute)) || result.ExpiresAt.After(a.now().Add(65*time.Minute)) { - return brokerIssued{}, errBroker + path := "/app/installations/" + strconv.FormatInt(approval.InstallationID, 10) + "/access_tokens" + if readOnly { + err = a.callWithBeforeRequest(ctx, "POST", path, "Bearer "+jwt, body, 201, &result, beforeRequest) + } else { + err = a.call(ctx, "POST", path, "Bearer "+jwt, body, 201, &result) } - if (readOnly && !result.matchesReadOnlyScope(approval)) || (!readOnly && !result.matchesScope(approval)) { + if err != nil { + return readOnlyRevocationCandidate(result, readOnly), errBroker + } + if !validBrokerToken(result.Token) || len(result.Token) > 1024 { return brokerIssued{}, errBroker } + if !result.ExpiresAt.After(a.now().Add(time.Minute)) || result.ExpiresAt.After(a.now().Add(65*time.Minute)) || + (readOnly && !result.matchesReadOnlyScope(approval)) || (!readOnly && !result.matchesScope(approval)) { + return readOnlyRevocationCandidate(result, readOnly), errBroker + } return result, nil } + +// A syntactically valid token returned by the read-only endpoint must remain +// available to the caller for cleanup even when the response fails a later +// expiry or scope check. Management-token validation never exposes a cleanup +// candidate because that credential has a different authority boundary. +func readOnlyRevocationCandidate(result brokerIssued, readOnly bool) brokerIssued { + if !readOnly || !validBrokerToken(result.Token) || len(result.Token) > 1024 { + return brokerIssued{} + } + return brokerIssued{Token: result.Token} +} func (a *brokerAPI) preflight(ctx context.Context, approval BrokerApproval, token string) error { if approval.ManagementScope != "" && approval.ManagementScope != organizationOnlyManagementScope { return errBroker diff --git a/experiments/g02-auth/broker_entry.go b/experiments/g02-auth/broker_entry.go index dd82c09b..b6dda040 100644 --- a/experiments/g02-auth/broker_entry.go +++ b/experiments/g02-auth/broker_entry.go @@ -292,7 +292,7 @@ func runBrokerWithPrivateHandoff(ctx context.Context, files BrokerFiles, input * } else if files.ControllerBinary != "" || files.ControllerApproval != "" || files.ControllerStateDirectory != "" || files.WorkerApproval != "" || files.WorkerStateDirectory != "" || approval.ControllerBinarySHA256 != "" || approval.ControllerApprovalSHA256 != "" || approval.ControllerHarnessSHA != "" { return BrokerResult{}, errBroker } - attempt, err := openBrokerJournal(files.StateDirectory, approval) + attempt, err := openBrokerJournalForExecution(ctx, files.StateDirectory, approval) if err != nil { return BrokerResult{}, errBroker } diff --git a/experiments/g02-auth/broker_journal.go b/experiments/g02-auth/broker_journal.go index 567fd0de..21f66f31 100644 --- a/experiments/g02-auth/broker_journal.go +++ b/experiments/g02-auth/broker_journal.go @@ -1,6 +1,7 @@ package enrollment import ( + "context" "crypto/sha256" "encoding/hex" "encoding/json" @@ -14,6 +15,7 @@ type brokerJournal struct { root *os.Root path string directoryInfo, fileInfo os.FileInfo + syncFile func(phase string) error } // openBrokerJournal durably claims one exact attempt path and holds its @@ -75,6 +77,28 @@ func openBrokerJournal(path string, a BrokerApproval) (j *brokerJournal, err err } return j, nil } + +type brokerJournalSyncFailureHookKey struct{} + +type brokerJournalSyncFailureHook func(*brokerJournal, string) error + +// openBrokerJournalForExecution keeps test-only sync fault injection scoped to +// one execution context. Production contexts carry no hook and use the +// canonical journal opener. +func openBrokerJournalForExecution(ctx context.Context, path string, a BrokerApproval) (*brokerJournal, error) { + if ctx == nil { + return nil, errBroker + } + journal, err := openBrokerJournal(path, a) + if err != nil { + return nil, err + } + if hook, ok := ctx.Value(brokerJournalSyncFailureHookKey{}).(brokerJournalSyncFailureHook); ok && hook != nil { + journal.syncFile = func(phase string) error { return hook(journal, phase) } + } + return journal, nil +} + func (j *brokerJournal) append(phase string, metadata map[string]any) error { record := struct { Phase string `json:"phase"` @@ -85,7 +109,11 @@ func (j *brokerJournal) append(phase string, metadata map[string]any) error { if n, err := j.file.Write(data); err != nil || n != len(data) { return errBroker } - if j.file.Sync() != nil { + if j.syncFile != nil { + if j.syncFile(phase) != nil { + return errBroker + } + } else if j.file.Sync() != nil { return errBroker } return nil diff --git a/experiments/g02-auth/broker_management_scope_test.go b/experiments/g02-auth/broker_management_scope_test.go index 6bd2ed41..8cd4c0f7 100644 --- a/experiments/g02-auth/broker_management_scope_test.go +++ b/experiments/g02-auth/broker_management_scope_test.go @@ -143,7 +143,7 @@ func TestBrokerReadOnlyMintRequiresExactScope(t *testing.T) { } return response(http.StatusCreated, string(data)), nil })} - _, err = api.mintReadOnly(context.Background(), a, credential) + _, err = api.mintReadOnlyWithIntent(context.Background(), a, credential, func() error { return nil }) if (err == nil) != (fault == "") || fixture.tokenCalls != 1 { t.Fatalf("read-only mint accepted=%v fault=%s requests=%d", err == nil, fault, fixture.tokenCalls) } @@ -179,7 +179,7 @@ func TestBrokerReadOnlyOrganizationScopeOmitsRepositoryAccess(t *testing.T) { } return response(http.StatusCreated, string(payload)), nil })} - if _, err := api.mintReadOnly(context.Background(), a, credential); err != nil || fixture.tokenCalls != 1 { + if _, err := api.mintReadOnlyWithIntent(context.Background(), a, credential, func() error { return nil }); err != nil || fixture.tokenCalls != 1 { t.Fatalf("valid organization-only read token was refused: err=%v requests=%d", err, fixture.tokenCalls) } } diff --git a/experiments/g02-auth/broker_test.go b/experiments/g02-auth/broker_test.go index 39c888ea..14c92f7b 100644 --- a/experiments/g02-auth/broker_test.go +++ b/experiments/g02-auth/broker_test.go @@ -3,6 +3,7 @@ package enrollment import ( "context" "encoding/json" + "errors" "io" "net/http" "os" @@ -27,17 +28,23 @@ func TestPairedTerminalBrokerApprovalUsesDedicatedMode(t *testing.T) { } type brokerHTTPFixture struct { - t *testing.T - calls []string - events []string - tokenCalls int - mintRequests []brokerMintRequestFixture - badGroup bool - badGroupPolicy bool - mintError bool - token string - root string - admissionRoot string + t *testing.T + calls []string + events []string + tokenCalls int + mintRequests []brokerMintRequestFixture + badGroup bool + badGroupPolicy bool + mintError bool + revokeError bool + readTokenFault string + readTokenToken string + loseReadTokenResponse bool + readTokenAccepted bool + requireReadTokenLifecycle bool + token string + root string + admissionRoot string } type brokerMintRequestFixture struct { @@ -74,6 +81,13 @@ func (f *brokerHTTPFixture) RoundTrip(r *http.Request) (*http.Response, error) { if json.Unmarshal(body, &request) != nil || !reflect.DeepEqual(request.RepositoryIDs, []int64{501}) || len(request.Permissions) != 2 || request.Permissions["metadata"] != "read" || (request.Permissions["organization_self_hosted_runners"] != "read" && request.Permissions["organization_self_hosted_runners"] != "write") { f.t.Error("token request not narrowly scoped") } + readOnly := request.Permissions["organization_self_hosted_runners"] == "read" + if readOnly && f.requireReadTokenLifecycle { + admission, err := os.ReadFile(filepath.Join(f.admissionRoot, "broker-admission.jsonl")) + if err != nil || !strings.Contains(string(admission), `"kind":"read_token_lifecycle_started"`) { + f.t.Error("read-only token request preceded its durable account-level lifecycle intent") + } + } if request.Permissions["organization_self_hosted_runners"] == "write" && strings.Contains(string(data), "read_only_token_verified") && !strings.Contains(string(data), "management_token_request_started") { f.t.Error("management token request preceded its durable intent") } @@ -87,10 +101,39 @@ func (f *brokerHTTPFixture) RoundTrip(r *http.Request) (*http.Response, error) { } status = 201 token := f.token + expiresAt := time.Now().Add(time.Hour) + responsePermissions := request.Permissions if request.Permissions["organization_self_hosted_runners"] == "read" { + f.readTokenAccepted = true token = "synthetic-private-read-installation-token" + if f.readTokenToken != "" { + token = f.readTokenToken + } + switch f.readTokenFault { + case "write-scope": + responsePermissions = map[string]string{"organization_self_hosted_runners": "write", "metadata": "read"} + case "expired": + expiresAt = time.Now().Add(-time.Minute) + } + } + payload = map[string]any{"token": token, "expires_at": expiresAt, "permissions": responsePermissions, "repository_selection": "selected", "repositories": []any{repo}} + if readOnly && f.loseReadTokenResponse { + return nil, errors.New("synthetic accepted read-token response lost") + } + case "/installation/token": + if r.Method != http.MethodDelete || r.Header.Get("Authorization") != "Bearer synthetic-private-read-installation-token" { + f.t.Error("installation-token revocation used the wrong method or authority") + } + data, err := os.ReadFile(filepath.Join(f.root, "broker.jsonl")) + if err != nil || !strings.Contains(string(data), "read_only_token_revocation_started") { + f.t.Error("installation-token revocation preceded its durable journal intent") + } + if f.revokeError { + status = http.StatusBadGateway + payload = map[string]string{"message": "synthetic revocation failure"} + } else { + status = http.StatusNoContent } - payload = map[string]any{"token": token, "expires_at": time.Now().Add(time.Hour), "permissions": request.Permissions, "repository_selection": "selected", "repositories": []any{repo}} case "/installation/repositories": payload = map[string]any{"total_count": 1, "repositories": []any{repo}} case "/repos/org-a/canary": @@ -112,6 +155,9 @@ func (f *brokerHTTPFixture) RoundTrip(r *http.Request) (*http.Response, error) { default: f.t.Fatal("unapproved API call") } + if status == http.StatusNoContent { + return response(status, ""), nil + } data, _ := json.Marshal(payload) return response(status, string(data)), nil } @@ -127,6 +173,85 @@ func TestBrokerWorkflowVerificationUsesHeadBranchForAttestedBranchRef(t *testing } } +func TestBrokerReadOnlyRevocationRecordsUncertainWhenRevokedSyncFails(t *testing.T) { + a, _, fixtureAPI, fixture, root := newBrokerFixture(t) + journal, err := openBrokerJournal(root, a) + if err != nil { + t.Fatalf("open attempt journal: %v", err) + } + t.Cleanup(journal.close) + claim, err := openBrokerAdmission(fixture.admissionRoot, a, journal, nil, fixtureAPI.syncDirectory) + if err != nil { + t.Fatalf("open admission claim: %v", err) + } + t.Cleanup(claim.close) + if err := claim.beginReadTokenLifecycle(); err != nil { + t.Fatalf("record read-token lifecycle intent: %v", err) + } + journal.syncFile = func(phase string) error { + if phase == "read_only_token_revoked" { + return errors.New("synthetic fsync failure") + } + return journal.file.Sync() + } + deleteCalls := 0 + api := newBrokerAPI(time.Now, transportFunc(func(r *http.Request) (*http.Response, error) { + deleteCalls++ + if r.Method != http.MethodDelete || r.Header.Get("Authorization") != "Bearer synthetic-private-read-installation-token" { + t.Errorf("revocation used the wrong method or authority: %s %s", r.Method, r.URL.Path) + } + return response(http.StatusNoContent, ""), nil + })) + if err := revokeBrokerReadOnlyToken(context.Background(), api, journal, claim, "synthetic-private-read-installation-token", "synthetic-private-workflow-token"); err == nil { + t.Fatal("revocation succeeded despite a failed durable success record") + } + if deleteCalls != 1 { + t.Fatalf("revocation requests=%d, want exactly one", deleteCalls) + } + data, err := os.ReadFile(filepath.Join(root, "broker.jsonl")) + if err != nil { + t.Fatal(err) + } + started := strings.Index(string(data), "read_only_token_revocation_started") + revoked := strings.Index(string(data), "read_only_token_revoked") + uncertain := strings.Index(string(data), "read_only_token_revocation_uncertain") + if started < 0 || revoked <= started || uncertain <= revoked || !strings.Contains(string(data), "revocation_result_persistence_failed") { + t.Fatalf("failed revoked sync was not followed by a terminal uncertainty record: %s", data) + } + if strings.Contains(string(data), "synthetic-private-read-installation-token") { + t.Fatal("read-only installation token was persisted in the broker journal") + } +} + +func TestBrokerReadOnlyMintDoesNotSendWhenLifecycleIntentCannotSync(t *testing.T) { + a, credential, api, fixture, root := newBrokerFixture(t) + journal, err := openBrokerJournal(root, a) + if err != nil { + t.Fatalf("open attempt journal: %v", err) + } + t.Cleanup(journal.close) + claim, err := openBrokerAdmission(fixture.admissionRoot, a, journal, nil, api.syncDirectory) + if err != nil { + t.Fatalf("open admission claim: %v", err) + } + t.Cleanup(claim.close) + claim.sync = func(*os.Root) error { return errors.New("synthetic admission fsync failure") } + parsedCredential, err := parseCredential(credential) + if err != nil { + t.Fatalf("parse synthetic App credential: %v", err) + } + if issued, err := api.mintReadOnlyWithIntent(context.Background(), a, parsedCredential, nil); err == nil || issued.Token != "" || fixture.tokenCalls != 0 || len(fixture.calls) != 0 { + t.Fatalf("read-only mint allowed a missing lifecycle callback: issued=%v err=%v mints=%d calls=%v", issued, err, fixture.tokenCalls, fixture.calls) + } + + if issued, err := api.mintReadOnlyWithIntent(context.Background(), a, parsedCredential, claim.beginReadTokenLifecycle); err == nil || issued.Token != "" { + t.Fatalf("read-only mint crossed a failed lifecycle intent: issued=%v err=%v", issued, err) + } + if fixture.tokenCalls != 0 || len(fixture.calls) != 0 { + t.Fatalf("read-token request dispatched before durable account intent: mints=%d calls=%v", fixture.tokenCalls, fixture.calls) + } +} + func TestBrokerWorkflowRunHeadBranchMatchesOnlySupportedRefs(t *testing.T) { for _, tc := range []struct { name, headBranch, workflowRef string diff --git a/experiments/g02-auth/broker_v3_final_integration_test.go b/experiments/g02-auth/broker_v3_final_integration_test.go index 36b34ad7..b51478bc 100644 --- a/experiments/g02-auth/broker_v3_final_integration_test.go +++ b/experiments/g02-auth/broker_v3_final_integration_test.go @@ -5,6 +5,7 @@ import ( "context" "crypto/ed25519" "encoding/json" + "errors" "io" "net/http" "os" @@ -31,6 +32,7 @@ func newBrokerV3RunFixture(t *testing.T, phase string, changedCallee bool, rootM a, candidate, _, fixture, attempt := newBrokerFixture(t) a.Mode, a.Phase = "controller", phase a.GroupPolicy = groupPolicyFixture() + fixture.requireReadTokenLifecycle = true a.AllowVerificationAuthority = phase == "after-ack" now := time.Now() a.ExpiresAt = now.Add(time.Hour) @@ -160,6 +162,29 @@ func newBrokerV3RunFixture(t *testing.T, phase string, changedCallee bool, rootM return brokerV3RunFixture{files: BrokerFiles{ApprovalPath: approvalPath, StateDirectory: attempt, ControllerBinary: binary.path, ControllerApproval: controllerPath, ControllerStateDirectory: controllerState}, candidate: candidate, fixture: fixture, workflowGET: workflowGET, changeCalleeAt: changeCalleeAt} } +func brokerJournalSyncFailureContext(ctx context.Context, failedPhases ...string) context.Context { + failed := make(map[string]bool, len(failedPhases)) + for _, phase := range failedPhases { + failed[phase] = true + } + return context.WithValue(ctx, brokerJournalSyncFailureHookKey{}, brokerJournalSyncFailureHook(func(journal *brokerJournal, phase string) error { + if failed[phase] { + return errors.New("synthetic fsync failure") + } + return journal.file.Sync() + })) +} + +func brokerFixtureCallCount(calls []string, want string) int { + count := 0 + for _, call := range calls { + if call == want { + count++ + } + } + return count +} + func TestBrokerExperimentV3ReadOnlyPreflightRejectsBeforeWriteMint(t *testing.T) { fixture := newBrokerV3RunFixture(t, "create", false) fixture.fixture.badGroupPolicy = true @@ -179,6 +204,12 @@ func TestBrokerExperimentV3ReadOnlyPreflightRejectsBeforeWriteMint(t *testing.T) if len(fixture.fixture.mintRequests) != 1 || !reflect.DeepEqual(fixture.fixture.mintRequests[0].RepositoryIDs, []int64{501}) || !reflect.DeepEqual(fixture.fixture.mintRequests[0].Permissions, wantPermissions) { t.Fatalf("ineligible group did not stop after an exact read-only token: requests=%+v", fixture.fixture.mintRequests) } + if !reflect.DeepEqual(fixture.fixture.calls[len(fixture.fixture.calls)-1:], []string{"DELETE /installation/token"}) { + t.Fatalf("read-only token was not revoked after rejected provider preflight: %v", fixture.fixture.calls) + } + if got := brokerFixtureCallCount(fixture.fixture.calls, "DELETE /installation/token"); got != 1 { + t.Fatalf("read-only token DELETE requests=%d, want exactly one: %v", got, fixture.fixture.calls) + } journal, err := os.ReadFile(filepath.Join(fixture.files.StateDirectory, "broker.jsonl")) if err != nil { t.Fatal(err) @@ -188,6 +219,271 @@ func TestBrokerExperimentV3ReadOnlyPreflightRejectsBeforeWriteMint(t *testing.T) } } +func TestBrokerExperimentV3StopsWhenReadOnlyTokenRevocationIsUncertain(t *testing.T) { + fixture := newBrokerV3RunFixture(t, "create", false) + fixture.fixture.revokeError = true + management, err := json.Marshal(brokerInput{PEM: string(fixture.candidate.PEM), VerificationToken: "synthetic-private-workflow-token"}) + if err != nil { + t.Fatal(err) + } + input := brokerV3Pipe(t, management) + observation := brokerV3Pipe(t, []byte("synthetic-private-workflow-token")) + workflowSeed := brokerV3Pipe(t, bytes.Repeat([]byte{1}, ed25519.SeedSize)) + issuerSeed := brokerV3Pipe(t, bytes.Repeat([]byte{71}, ed25519.SeedSize)) + result, runErr := RunBrokerExperimentWithInputs(context.Background(), fixture.files, input, observation, workflowSeed, issuerSeed) + if runErr == nil || result.Status != "" || fixture.fixture.tokenCalls != 1 || *fixture.workflowGET != 1 { + t.Fatalf("uncertain read-only-token revocation crossed the write boundary: result=%+v err=%v mints=%d workflow_gets=%d calls=%v", result, runErr, fixture.fixture.tokenCalls, *fixture.workflowGET, fixture.fixture.calls) + } + if !reflect.DeepEqual(fixture.fixture.calls[len(fixture.fixture.calls)-1:], []string{"DELETE /installation/token"}) { + t.Fatalf("read-only installation token was not revoked as the final API operation: %v", fixture.fixture.calls) + } + if got := brokerFixtureCallCount(fixture.fixture.calls, "DELETE /installation/token"); got != 1 { + t.Fatalf("read-only token DELETE requests=%d, want exactly one: %v", got, fixture.fixture.calls) + } + journal, err := os.ReadFile(filepath.Join(fixture.files.StateDirectory, "broker.jsonl")) + if err != nil { + t.Fatal(err) + } + if !bytes.Contains(journal, []byte("read_only_token_revocation_uncertain")) || bytes.Contains(journal, []byte("management_token_request_started")) || bytes.Contains(journal, []byte("controller_handoff_started")) { + t.Fatalf("uncertain revocation was not durably fenced before write mint: %s", journal) + } + if bytes.Contains(journal, []byte("synthetic-private-read-installation-token")) { + t.Fatal("read-only installation token was persisted in the broker journal") + } +} + +func TestBrokerExperimentV3RevokedRecordSyncFailureStopsBeforeWriteMint(t *testing.T) { + fixture := newBrokerV3RunFixture(t, "create", false) + management, err := json.Marshal(brokerInput{PEM: string(fixture.candidate.PEM), VerificationToken: "synthetic-private-workflow-token"}) + if err != nil { + t.Fatal(err) + } + ctx := brokerJournalSyncFailureContext(context.Background(), "read_only_token_revoked") + result, runErr := RunBrokerExperimentWithInputs( + ctx, fixture.files, + brokerV3Pipe(t, management), + brokerV3Pipe(t, []byte("synthetic-private-workflow-token")), + brokerV3Pipe(t, bytes.Repeat([]byte{1}, ed25519.SeedSize)), + brokerV3Pipe(t, bytes.Repeat([]byte{71}, ed25519.SeedSize)), + ) + if runErr == nil || result.Status != "" || fixture.fixture.tokenCalls != 1 || *fixture.workflowGET != 1 { + t.Fatalf("revoked-record sync failure crossed the write boundary: result=%+v err=%v mints=%d workflow_gets=%d calls=%v", result, runErr, fixture.fixture.tokenCalls, *fixture.workflowGET, fixture.fixture.calls) + } + if got := brokerFixtureCallCount(fixture.fixture.calls, "DELETE /installation/token"); got != 1 { + t.Fatalf("read-only token DELETE requests=%d, want exactly one: %v", got, fixture.fixture.calls) + } + if len(fixture.fixture.calls) == 0 || fixture.fixture.calls[len(fixture.fixture.calls)-1] != "DELETE /installation/token" { + t.Fatalf("revocation was not the last provider operation: %v", fixture.fixture.calls) + } + journal, err := os.ReadFile(filepath.Join(fixture.files.StateDirectory, "broker.jsonl")) + if err != nil { + t.Fatal(err) + } + text := string(journal) + started := strings.Index(text, "read_only_token_revocation_started") + revoked := strings.Index(text, "read_only_token_revoked") + uncertain := strings.Index(text, "read_only_token_revocation_uncertain") + if started < 0 || revoked <= started || uncertain <= revoked || strings.Contains(text, "management_token_request_started") || strings.Contains(text, "controller_handoff_started") { + t.Fatalf("sync failure was not fenced by a terminal uncertainty record before writes: %s", text) + } + if bytes.Contains(journal, []byte("synthetic-private-read-installation-token")) { + t.Fatal("read-only installation token was persisted in the broker journal") + } +} + +func TestBrokerExperimentV3DoubleRevocationJournalSyncFailureBlocksLaterInspect(t *testing.T) { + create := newBrokerV3RunFixture(t, "create", false) + _, runErr := executeBrokerV3Fixture(t, + brokerJournalSyncFailureContext(context.Background(), "read_only_token_revoked", "read_only_token_revocation_uncertain"), + create, + ) + if runErr == nil || create.fixture.tokenCalls != 1 || brokerFixtureCallCount(create.fixture.calls, "DELETE /installation/token") != 1 { + t.Fatalf("compound journal fault did not stop after one read-token DELETE: err=%v mints=%d calls=%v", runErr, create.fixture.tokenCalls, create.fixture.calls) + } + admission, err := os.ReadFile(filepath.Join(create.fixture.admissionRoot, "broker-admission.jsonl")) + if err != nil || !bytes.Contains(admission, []byte(`"kind":"read_token_lifecycle_started"`)) || bytes.Contains(admission, []byte(`"kind":"read_token_lifecycle_confirmed"`)) { + t.Fatalf("compound journal fault did not leave an unresolved account-level lifecycle intent: err=%v", err) + } + + inspect := newBrokerV3RunFixture(t, "inspect", false) + inspect.fixture.admissionRoot = create.fixture.admissionRoot + inspect.files.ControllerStateDirectory = create.files.ControllerStateDirectory + result, inspectErr := executeBrokerV3Fixture(t, context.Background(), inspect) + if inspectErr == nil || result.Status != "" || inspect.fixture.tokenCalls != 0 || brokerFixtureCallCount(inspect.fixture.calls, "DELETE /installation/token") != 0 { + t.Fatalf("later inspect crossed an unresolved read-token revocation: result=%+v err=%v mints=%d calls=%v", result, inspectErr, inspect.fixture.tokenCalls, inspect.fixture.calls) + } + for _, call := range inspect.fixture.calls { + if strings.HasPrefix(call, "POST /app/installations/") { + t.Fatalf("later inspect minted a token after unresolved revocation: %v", inspect.fixture.calls) + } + } +} + +func TestBrokerExperimentV3LostReadTokenMintResponseBlocksLaterInspect(t *testing.T) { + create := newBrokerV3RunFixture(t, "create", false) + create.fixture.loseReadTokenResponse = true + _, createErr := executeBrokerV3Fixture(t, context.Background(), create) + if createErr == nil || create.fixture.tokenCalls != 1 || !create.fixture.readTokenAccepted || brokerFixtureCallCount(create.fixture.calls, "DELETE /installation/token") != 0 { + t.Fatalf("fixture did not model an accepted read-token mint with a lost response: err=%v mints=%d accepted=%t calls=%v", createErr, create.fixture.tokenCalls, create.fixture.readTokenAccepted, create.fixture.calls) + } + admission, err := os.ReadFile(filepath.Join(create.fixture.admissionRoot, "broker-admission.jsonl")) + started := bytes.Contains(admission, []byte(`"kind":"read_token_lifecycle_started"`)) + confirmed := bytes.Contains(admission, []byte(`"kind":"read_token_lifecycle_confirmed"`)) + if err != nil || !started || confirmed { + t.Errorf("lost response account-level lifecycle state: err=%v started=%t confirmed=%t", err, started, confirmed) + } + + inspect := newBrokerV3RunFixture(t, "inspect", false) + inspect.fixture.admissionRoot = create.fixture.admissionRoot + inspect.files.ControllerStateDirectory = create.files.ControllerStateDirectory + result, inspectErr := executeBrokerV3Fixture(t, context.Background(), inspect) + if inspectErr == nil || result.Status != "" || inspect.fixture.tokenCalls != 0 || len(inspect.fixture.calls) != 0 { + t.Fatalf("later inspect crossed an unresolved read-token issuance: result=%+v err=%v mints=%d calls=%v", result, inspectErr, inspect.fixture.tokenCalls, inspect.fixture.calls) + } +} + +func TestBrokerExperimentV3LegacyIncompleteClaimWithoutLifecycleBlocksLaterInspect(t *testing.T) { + create := newBrokerV3RunFixture(t, "create", false) + if _, err := executeBrokerV3Fixture(t, context.Background(), create); err != nil { + t.Fatalf("complete synthetic v3 create: %v", err) + } + ledgerPath := filepath.Join(create.fixture.admissionRoot, "broker-admission.jsonl") + ledger, err := os.ReadFile(ledgerPath) + if err != nil { + t.Fatalf("read synthetic admission ledger: %v", err) + } + var legacyLines []string + for _, line := range strings.Split(strings.TrimSuffix(string(ledger), "\n"), "\n") { + if strings.Contains(line, `"kind":"read_token_lifecycle_`) || strings.Contains(line, `"kind":"complete"`) { + continue + } + legacyLines = append(legacyLines, line) + } + if len(legacyLines) != 2 { + t.Fatalf("expected a legacy header plus one incomplete claim, got %d ledger lines", len(legacyLines)) + } + if err := os.WriteFile(ledgerPath, []byte(strings.Join(legacyLines, "\n")+"\n"), 0600); err != nil { + t.Fatalf("write synthetic legacy incomplete ledger: %v", err) + } + + inspect := newBrokerV3RunFixture(t, "inspect", false) + inspect.fixture.admissionRoot = create.fixture.admissionRoot + inspect.files.ControllerStateDirectory = create.files.ControllerStateDirectory + result, inspectErr := executeBrokerV3Fixture(t, context.Background(), inspect) + if inspectErr == nil || result.Status != "" || inspect.fixture.tokenCalls != 0 || len(inspect.fixture.calls) != 0 { + t.Fatalf("legacy incomplete reusable-callee claim without lifecycle proof was recovered: result=%+v err=%v mints=%d calls=%v", result, inspectErr, inspect.fixture.tokenCalls, inspect.fixture.calls) + } +} + +func TestBrokerExperimentV3PreRequestFailureRecordsNoDispatchForLaterInspect(t *testing.T) { + create := newBrokerV3RunFixture(t, "create", false) + _, createErr := executeBrokerV3Fixture(t, brokerJournalSyncFailureContext(context.Background(), "token_request_started"), create) + if createErr == nil || create.fixture.tokenCalls != 0 || brokerFixtureCallCount(create.fixture.calls, "POST /app/installations/201/access_tokens") != 0 { + t.Fatalf("token-request journal failure reached issuance: err=%v mints=%d calls=%v", createErr, create.fixture.tokenCalls, create.fixture.calls) + } + admission, err := os.ReadFile(filepath.Join(create.fixture.admissionRoot, "broker-admission.jsonl")) + if err != nil || !bytes.Contains(admission, []byte(`"kind":"read_token_request_not_dispatched"`)) { + t.Fatalf("pre-request failure lacks durable no-dispatch proof: err=%v", err) + } + + inspect := newBrokerV3RunFixture(t, "inspect", false) + inspect.fixture.admissionRoot = create.fixture.admissionRoot + inspect.files.ControllerStateDirectory = create.files.ControllerStateDirectory + result, inspectErr := executeBrokerV3Fixture(t, context.Background(), inspect) + if inspectErr != nil || result.Status != "controller_completed" || inspect.fixture.tokenCalls != 2 { + t.Fatalf("durable no-dispatch proof failed to permit later inspect: result=%+v err=%v mints=%d calls=%v", result, inspectErr, inspect.fixture.tokenCalls, inspect.fixture.calls) + } +} + +func executeBrokerV3Fixture(t *testing.T, ctx context.Context, fixture brokerV3RunFixture) (BrokerResult, error) { + t.Helper() + management, err := json.Marshal(brokerInput{PEM: string(fixture.candidate.PEM), VerificationToken: "synthetic-private-workflow-token"}) + if err != nil { + t.Fatal(err) + } + return RunBrokerExperimentWithInputs( + ctx, fixture.files, + brokerV3Pipe(t, management), + brokerV3Pipe(t, []byte("synthetic-private-workflow-token")), + brokerV3Pipe(t, bytes.Repeat([]byte{1}, ed25519.SeedSize)), + brokerV3Pipe(t, bytes.Repeat([]byte{71}, ed25519.SeedSize)), + ) +} + +func TestBrokerExperimentV3RevokesTokenFromRejectedReadIssuance(t *testing.T) { + for _, fault := range []string{"write-scope", "expired"} { + t.Run(fault, func(t *testing.T) { + fixture := newBrokerV3RunFixture(t, "create", false) + fixture.fixture.readTokenFault = fault + management, err := json.Marshal(brokerInput{PEM: string(fixture.candidate.PEM), VerificationToken: "synthetic-private-workflow-token"}) + if err != nil { + t.Fatal(err) + } + input := brokerV3Pipe(t, management) + observation := brokerV3Pipe(t, []byte("synthetic-private-workflow-token")) + workflowSeed := brokerV3Pipe(t, bytes.Repeat([]byte{1}, ed25519.SeedSize)) + issuerSeed := brokerV3Pipe(t, bytes.Repeat([]byte{71}, ed25519.SeedSize)) + result, runErr := RunBrokerExperimentWithInputs(context.Background(), fixture.files, input, observation, workflowSeed, issuerSeed) + if runErr == nil || result.Status != "" || fixture.fixture.tokenCalls != 1 || *fixture.workflowGET != 1 { + t.Fatalf("rejected read-token issuance crossed its initial workflow gate: result=%+v err=%v mints=%d workflow_gets=%d calls=%v", result, runErr, fixture.fixture.tokenCalls, *fixture.workflowGET, fixture.fixture.calls) + } + if len(fixture.fixture.calls) == 0 || fixture.fixture.calls[len(fixture.fixture.calls)-1] != "DELETE /installation/token" { + t.Fatalf("usable rejected token was not revoked: calls=%v", fixture.fixture.calls) + } + if got := brokerFixtureCallCount(fixture.fixture.calls, "DELETE /installation/token"); got != 1 { + t.Fatalf("read-only token DELETE requests=%d, want exactly one: %v", got, fixture.fixture.calls) + } + for _, call := range fixture.fixture.calls { + if call == "GET /installation/repositories" || strings.HasPrefix(call, "GET /orgs/org-a/actions/runner-groups/") { + t.Fatalf("rejected token was used for provider preflight: calls=%v", fixture.fixture.calls) + } + } + journal, err := os.ReadFile(filepath.Join(fixture.files.StateDirectory, "broker.jsonl")) + if err != nil { + t.Fatal(err) + } + if !bytes.Contains(journal, []byte("read_only_token_revocation_started")) || !bytes.Contains(journal, []byte("read_only_token_revoked")) || bytes.Contains(journal, []byte("read_only_token_verified")) || bytes.Contains(journal, []byte("management_token_request_started")) || bytes.Contains(journal, []byte("controller_handoff_started")) { + t.Fatalf("rejected issuance cleanup crossed an invalid journal boundary: %s", journal) + } + if bytes.Contains(journal, []byte("synthetic-private-read-installation-token")) { + t.Fatal("rejected read-token candidate was persisted in the broker journal") + } + }) + } +} + +func TestBrokerExperimentV3DoesNotRevokeVerificationTokenCollision(t *testing.T) { + fixture := newBrokerV3RunFixture(t, "create", false) + fixture.fixture.readTokenToken = "synthetic-private-workflow-token" + management, err := json.Marshal(brokerInput{PEM: string(fixture.candidate.PEM), VerificationToken: "synthetic-private-workflow-token"}) + if err != nil { + t.Fatal(err) + } + input := brokerV3Pipe(t, management) + observation := brokerV3Pipe(t, []byte("synthetic-private-workflow-token")) + workflowSeed := brokerV3Pipe(t, bytes.Repeat([]byte{1}, ed25519.SeedSize)) + issuerSeed := brokerV3Pipe(t, bytes.Repeat([]byte{71}, ed25519.SeedSize)) + result, runErr := RunBrokerExperimentWithInputs(context.Background(), fixture.files, input, observation, workflowSeed, issuerSeed) + if runErr == nil || result.Status != "" || fixture.fixture.tokenCalls != 1 || *fixture.workflowGET != 1 { + t.Fatalf("verification-token collision was not rejected before use: result=%+v err=%v mints=%d workflow_gets=%d calls=%v", result, runErr, fixture.fixture.tokenCalls, *fixture.workflowGET, fixture.fixture.calls) + } + for _, call := range fixture.fixture.calls { + if call == "DELETE /installation/token" || call == "GET /installation/repositories" || strings.HasPrefix(call, "GET /orgs/org-a/actions/runner-groups/") { + t.Fatalf("caller verification token was used or revoked: calls=%v", fixture.fixture.calls) + } + } + journal, err := os.ReadFile(filepath.Join(fixture.files.StateDirectory, "broker.jsonl")) + if err != nil { + t.Fatal(err) + } + if !bytes.Contains(journal, []byte("read_only_token_revocation_skipped")) || !bytes.Contains(journal, []byte("verification_token_collision")) || bytes.Contains(journal, []byte("read_only_token_revocation_started")) { + t.Fatalf("verification-token collision outcome was not durably recorded without a revoke attempt: %s", journal) + } + if bytes.Contains(journal, []byte("synthetic-private-workflow-token")) { + t.Fatal("caller verification token was persisted in the broker journal") + } +} + func brokerV3Pipe(t *testing.T, data []byte) *os.File { t.Helper() r, w, err := os.Pipe() @@ -283,6 +579,7 @@ func TestBrokerExperimentV3FinalGETFencesCalleeBeforeMintAndHandoff(t *testing.T "GET /repos/org-a/canary", "GET /orgs/org-a/actions/runner-groups/3", "GET /orgs/org-a/actions/runner-groups/3/repositories", + "DELETE /installation/token", "GET /repos/org-a/canary/actions/runs/7", "POST /app/installations/201/access_tokens", "GET /installation/repositories",