diff --git a/BACKLOG.md b/BACKLOG.md index 4645a35..20c4c69 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -11,7 +11,7 @@ Loose ideas, not yet scheduled. Anything with a shape clear enough to plan again the band-aid impact table. Reachable from ② Review and Setup — **Open HTML report ↗** (rebuilt from the current out dir on every request, so it is never stale) and **Download** for a stamped `vpcopilot-report--.html`. Not included: the benchmark scorecard — that still lives - only in the console's ⑥ Benchmark step. Original ask: a standalone, static, shareable export of + only in the console's ⑦ Benchmark step. Original ask: a standalone, static, shareable export of a run for stakeholders. _(Requested 2026-07-01.)_ - ~~**Ops console admin panel (localhost).**~~ ✅ **Done** in the console MVP — the ⚙ Setup page reads/writes the local `.env` (XC creds + model API keys), redacting secrets. diff --git a/README.md b/README.md index 5425fe5..d726b1b 100644 --- a/README.md +++ b/README.md @@ -84,6 +84,7 @@ cp .env.example .env # model key(s) + XC creds + GITHUB_TOK vpcopilot console # scan, apply, PR, retire — all from the UI # or headless: vpcopilot scan /path/to/app-repo --out out +vpcopilot simulate --out out --from-tenant # blast radius before you apply vpcopilot export --out out # evidence bundle (.zip) of every change made to an LB ``` @@ -95,32 +96,41 @@ reference: **[docs/USAGE.md](docs/USAGE.md)**. ## The console A guided flow that follows the lifecycle — a persistent hero band (N exploitable → mitigated live -in seconds vs. change-control days) sits on top of six steps: +in seconds vs. change-control days) sits on top of seven steps: 1. **Scan** — point at a repo; read-only, safe. The log holds the whole transcript in a scrollable box, so a long run can be read end-to-end while it's still going. 2. **Review** — findings + the recommended XC control; click a row to inspect exploit / code / policy. -3. **Mitigate** — apply each band-aid live; the refiner streams `before 200 → after 403 BLOCKED` +3. **Simulate** — replay a recorded traffic sample against each candidate through a spare LB and see + what it *would* block before anything touches production. A policy over the false-positive + threshold warns at the gate and needs an explicit, audited override. +4. **Mitigate** — apply each band-aid live; the refiner streams `before 200 → after 403 BLOCKED` with a *self-healed in N attempts* / *unfixable → ship the code fix* badge. -4. **Cure** — open the code-fix PR for each finding. -5. **Retire** — the four-state ledger track, and the **audit trail** of every change made to a load +5. **Cure** — open the code-fix PR for each finding. +6. **Retire** — the four-state ledger track, and the **audit trail** of every change made to a load balancer — exportable as an evidence bundle. -6. **Benchmark** — build a model-tagged report from this run, then compare models side by side. +7. **Benchmark** — build a model-tagged report from this run, then compare models side by side. The shareable HTML report opens (or downloads) from **Review** and from **Setup** — it is rebuilt from the current run dir on every open, so it's always the latest run. Credentials, XC status, and the per-agent model wiring live under **Setup**. -**③ Mitigate** — apply each band-aid and watch it validate: +**③ Simulate** — replay recorded traffic against each candidate and see what it *would* block +before anything is applied. Here the SQLi service policy is surgical: 3 blocks in 500 recorded +requests, all of them attacks on the login endpoint: -![Mitigate step](docs/images/3-mitigate.png) +![Simulate step — blast radius](docs/images/3-simulate.png) -**⑤ Retire** — the four-state ledger (here `crapi-sqli-001` walked all the way to *retired*), and +**④ Mitigate** — apply each band-aid and watch it validate: + +![Mitigate step](docs/images/4-mitigate.png) + +**⑥ Retire** — the four-state ledger (here `crapi-sqli-001` walked all the way to *retired*), and under it the audit trail: each change tied to the vulnerability that justified it, the LB and XC namespace it touched, whether it's still live, and who ran it. Dry runs are absent by design — nothing changed, so there is nothing to answer for. -![Retire step — ledger and audit trail](docs/images/5-retire.png) +![Retire step — ledger and audit trail](docs/images/6-retire.png) Every scan also drops a standalone, shareable **`report.html`** — the same hero plus at-a-glance bars, the self-heal (`200 → 403`, *self-healed ×2*), the rate-limit behavioral proof, and the diff --git a/ROADMAP.md b/ROADMAP.md index db3aba0..3fb121d 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -89,17 +89,25 @@ Live validation on the LB stays. This runs before it, not instead of it. artifact — the same class of defect `lint_service_policy` shipped with until `artifact_spec()` (`apply.py:106-113`) was added to unwrap `{metadata, spec}`. -- [ ] **G2** Shadow simulation report. (M, P1) **Does not depend on G1** — replays through a - monitor-mode LB on the tenant (decision of 2026-07-27). +- [x] **G2** Shadow simulation report. (M, P1) — **DONE:** `simulate.py` + `traffic.py` replay a + recorded sample against each candidate on a spare LB and report the would-block set; + `vpcopilot simulate` / `GET`+`POST /api/simulate` / a ③ Simulate console step / a **Blast radius** + panel in the HTML report / `simulation.json` in the evidence bundle. Live on `vpcopilot-lab`: + the negative-amount policy blocked **1 of 200** recorded requests — the exploit — and **none** of + the 60 legitimate transfers (0.5%, under threshold); a deliberately overbroad `/api` DENY blocked + **154 of 200** (77%) and tripped it. 34 tests. Replay a traffic sample against candidate policies and report the would-block set before anything reaches the gate. - Per policy: requests evaluated, would-block count and rate, a sample of blocked requests, top blocked paths and user agents. A configurable threshold marking a policy `blocked_promotion` with the reason recorded. - Acceptance: the Nimbus negative-amount policy blocks the exploit and zero recorded - legitimate transfers; a deliberately overbroad policy trips the threshold and cannot be - promoted to apply; the result renders in the HTML report next to the band-aid impact - table; simulation runs with no XC credentials present. + legitimate transfers ✅; a deliberately overbroad policy trips the threshold ✅ and **warns + at the gate with an audited override** rather than being unpromotable (decision of + 2026-07-27 — a machine veto would invert "a human approves"); the result renders in the HTML + report next to the band-aid impact table ✅; **ingest** runs with no XC credentials, but the + would-block numbers need the tenant ✅ (rewritten from "simulation runs with no XC + credentials present", which only held for the deferred offline evaluator). - **Record sources (decided 2026-07-27): both.** `traffic.py` ingests a file (HAR or JSONL) **and** reads XC request logs directly, which needs a new read-only method on `xc.py` alongside the existing config-API calls. File ingest works with no tenant; the log read needs @@ -108,15 +116,25 @@ Live validation on the LB stays. This runs before it, not instead of it. `src/vpcopilot/xc.py`, `vpcopilot simulate [--logs | --from-tenant --lb --since ]`, `GET /api/simulate` + `POST /api/simulate`, - a Simulate step in the console between ② Review and ③ Mitigate. + a Simulate step in the console between ② Review and ④ Mitigate. - **Reconciled:** there is no "Apply" step — the console steps are Scan · Review · Mitigate · Cure · Retire · Benchmark. Inserting Simulate renumbers Mitigate through Benchmark, and registration touches four places in `console/static/index.html` (`STEPS`, the `
`, - the `show()` branch, the deep-link allowlist). Simulate is read-only, so it does **not** - join `ACTION_STEPS`. - - **Missing deliverable:** the acceptance requires a recorded legitimate-traffic sample and no - such fixture exists in the repo. `bench/fixtures/traffic/nimbus-sample.jsonl` (or equivalent) - has to ship with the item or the criterion is untestable. + the `show()` branch, the deep-link allowlist). **Corrected during build:** Simulate *is* in + `ACTION_STEPS` after all — it needs the LB and validate-URL fields from Run settings to know + which spare LB to replay through. It stays read-only with respect to the run's artifacts. + - **Deliverable shipped:** `bench/fixtures/traffic/nimbus-sample.jsonl` — 200 records, one + negative-amount transfer among 60 legitimate ones and 139 reads. + - **Three defects only the live run could find**, each now pinned by a test: + (1) attaching and replaying immediately counted an *unenforced* policy as harmless — 0/200 + blocked, including the exploit; a canary built from the finding's own probe now confirms + enforcement before anything is counted, and an unconfirmed run says so instead of reading as + safe. (2) one transient TLS error aborted a whole policy's simulation; failures are now + per-request, excluded from the rate rather than guessed either way. (3) attaching by name + assumed the policy object already existed — a fresh candidate has none, and referencing a + missing policy broke the LB config outright (every request failed in transit, not just the + matched ones); simulation now creates a throwaway `-vpcsim` object and deletes it, so it + never rides on, or mutates, the operator's own policies. - **Criterion rewritten** (G1 deferred, so replay runs on the tenant): *without XC credentials `vpcopilot simulate` parses and reports the record set and exits cleanly, evaluating nothing; it never errors for want of a tenant.* The would-block numbers require a monitor-mode LB. @@ -132,7 +150,7 @@ Live validation on the LB stays. This runs before it, not instead of it. convenience: it needs its own test file, a redaction count in `simulation.json`, and a `caveats` line in `export.build_manifest` stating what was stripped and what was not. -- [ ] **G3** Simulation feeds the refiner. (S, P2) Depends on G2. +- [ ] **G3** Simulation feeds the refiner. (S, P2) Depends on G2 (landed). `refiner.py` retries until the band-aid blocks the exploit. Add the second half: retry until it blocks the exploit **and** stays under the false-positive threshold. Today a refinement widening the policy to catch the exploit has nothing telling it it went too far. @@ -281,11 +299,11 @@ holds only while a human keeps checking. - Acceptance: `vpcopilot patches list` shows every live band-aid with age, TTL remaining, and PR state (`vpcopilot patches-list`); reconcile is idempotent and safe to run from cron or CI; an escalation - leaves the control in place and writes an audit record; the console's ⑤ Retire step + leaves the control in place and writes an audit record; the console's ⑥ Retire step surfaces TTL and escalation state. - Surfaces: extend `ledger.py` and `retire.py`, `vpcopilot reconcile` + `POST /api/reconcile`, `vpcopilot patches-list` + `GET /api/patches`. - - **Reconciled:** there is no Ledger tab — the ledger renders inside the ⑤ Retire step + - **Reconciled:** there is no Ledger tab — the ledger renders inside the ⑥ Retire step (`#ledgerBox`). Effort raised to **L**: TTL persistence, a three-outcome state machine, re-running probes against origin, two commands, two console surfaces, and cron idempotence. `patches list` as two words is not addable — there is no typer sub-app (`cli.py:15`, every diff --git a/bench/fixtures/traffic/nimbus-sample.jsonl b/bench/fixtures/traffic/nimbus-sample.jsonl new file mode 100644 index 0000000..64deaf5 --- /dev/null +++ b/bench/fixtures/traffic/nimbus-sample.jsonl @@ -0,0 +1,200 @@ +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "NimbusApp/2.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": -250}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:00:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 10.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:00:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "NimbusApp/2.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 17.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:01:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 25.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:02:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "curl/8.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 32.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:03:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 40.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:04:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "NimbusApp/2.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 47.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:05:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 55.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:06:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "curl/8.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 62.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:07:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 70.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:08:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "NimbusApp/2.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 77.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:09:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 85.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:10:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "curl/8.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 92.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:11:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 100.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:12:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "NimbusApp/2.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 107.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:13:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 115.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:14:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "curl/8.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 122.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:15:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 130.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:16:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "NimbusApp/2.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 137.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:17:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 145.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:18:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "curl/8.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 152.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:19:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 160.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:20:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "NimbusApp/2.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 167.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:21:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 175.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:22:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "curl/8.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 182.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:23:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 190.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:24:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "NimbusApp/2.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 197.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:25:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 205.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:26:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "curl/8.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 212.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:27:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 220.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:28:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "NimbusApp/2.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 227.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:29:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 235.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:30:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "curl/8.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 242.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:31:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 250.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:32:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "NimbusApp/2.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 257.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:33:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 265.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:34:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "curl/8.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 272.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:35:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 280.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:36:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "NimbusApp/2.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 287.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:37:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 295.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:38:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "curl/8.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 302.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:39:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 310.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:40:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "NimbusApp/2.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 317.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:41:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 325.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:42:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "curl/8.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 332.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:43:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 340.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:44:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "NimbusApp/2.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 347.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:45:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 355.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:46:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "curl/8.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 362.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:47:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 370.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:48:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "NimbusApp/2.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 377.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:49:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 385.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:50:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "curl/8.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 392.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:51:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 400.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:52:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "NimbusApp/2.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 407.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:53:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 415.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:54:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "curl/8.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 422.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:55:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 430.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:56:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "NimbusApp/2.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 437.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:57:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 445.0}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:58:00Z"} +{"method": "POST", "path": "/api/pay", "headers": {"User-Agent": "curl/8.1"}, "json_body": {"from_account": 1, "to_account_number": "4001 2233 0002", "amount": 452.5}, "status": 200, "source": "fixture", "ts": "2026-07-27T09:59:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:00:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:01:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:02:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:03:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:04:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:05:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:06:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:07:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:08:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:09:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:10:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:11:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:12:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:13:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:14:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:15:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:16:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:17:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:18:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:19:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:20:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:21:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:22:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:23:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:24:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:25:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:26:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:27:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:28:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:29:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:30:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:31:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:32:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:33:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:34:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:35:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:36:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:37:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:38:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:39:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:40:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:41:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:42:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:43:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:44:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:45:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:46:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:47:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:48:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:49:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:50:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:51:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:52:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:53:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:54:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:55:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:56:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:57:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:58:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:59:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:00:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:01:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:02:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:03:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:04:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:05:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:06:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:07:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:08:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:09:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:10:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:11:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:12:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:13:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:14:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:15:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:16:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:17:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:18:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:19:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:20:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:21:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:22:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:23:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:24:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:25:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:26:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:27:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:28:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:29:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:30:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:31:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:32:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:33:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:34:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:35:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:36:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:37:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:38:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:39:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:40:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:41:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:42:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:43:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:44:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:45:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:46:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:47:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:48:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:49:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:50:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:51:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:52:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:53:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:54:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:55:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:56:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:57:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:58:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:59:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:00:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:01:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:02:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:03:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:04:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:05:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:06:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:07:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:08:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:09:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:10:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:11:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:12:00Z"} +{"method": "GET", "path": "/api/statements", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:13:00Z"} +{"method": "GET", "path": "/api/profile", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:14:00Z"} +{"method": "GET", "path": "/", "headers": {"User-Agent": "curl/8.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:15:00Z"} +{"method": "GET", "path": "/api/fx", "headers": {"User-Agent": "Mozilla/5.0 (Macintosh)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:16:00Z"} +{"method": "GET", "path": "/health", "headers": {"User-Agent": "NimbusApp/2.1"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:17:00Z"} +{"method": "GET", "path": "/api/accounts", "headers": {"User-Agent": "Mozilla/5.0 (Windows)"}, "status": 200, "source": "fixture", "ts": "2026-07-27T10:18:00Z"} diff --git a/demo/build_demo_out.py b/demo/build_demo_out.py index b0490fc..6c8d463 100644 --- a/demo/build_demo_out.py +++ b/demo/build_demo_out.py @@ -197,6 +197,35 @@ def main(): control="service_policy", lb=LB, forced=False) _curate_identity() + # G2) a curated blast-radius result, so the offline demo shows the Simulate step doing its job: + # the SQLi service policy is surgical — it blocks the attack and nothing a customer does. + # Only service_policy is replayable (the other six are parameterized by the engine, not matched + # per request), so one policy is the honest picture for this dataset. + (OUT / "simulation.json").write_text(json.dumps({ + "ts": runmeta.utc_now(), "lb": LB, "source": f"xc:{LB}", + "records": 500, "records_replayed": 500, + "window": "2026-07-20T09:00:00Z..2026-07-20T10:00:00Z", + "redacted": {"authorization": 500, "cookie": 431, "password": 12}, + "bodies_available": True, + "policies": [{ + "policy_name": "deny-login-sqli", "control": "service_policy", + "finding_id": "crapi-sqli-001", "evaluated": 500, "would_block": 3, + "block_rate": 0.006, "threshold": 0.01, "blocked_promotion": False, "reason": "", + "errored": 0, "enforcement_confirmed": True, + "samples": [{"method": "POST", "path": "/identity/api/auth/login", "source": "xc", + "json_body": None, "user_agent": "python-requests/2.31", + "ts": "2026-07-20T09:14:02Z", "status": 200}], + "top_paths": [["/identity/api/auth/login", 3]], + "top_user_agents": [["python-requests/2.31", 2], ["sqlmap/1.7", 1]], + }], + "caveats": [ + "A would-block count describes THIS sample only — a quiet window makes any policy look " + "safe. The window and record count are stated above; read them together with the rate.", + "Replay runs one policy at a time on a spare LB. Two candidates share FIRST_MATCH " + "ordering, so a batched result would not describe either band-aid alone.", + ], + }, indent=2)) + from vpcopilot.report import write_report path = write_report(str(OUT)) print(f"wrote curated demo out/ -> {OUT}") diff --git a/demo/out/audit.log b/demo/out/audit.log index 1d08f94..5af355b 100644 --- a/demo/out/audit.log +++ b/demo/out/audit.log @@ -1,13 +1,13 @@ -{"ts": "2026-07-27T01:40:25.195575+00:00", "action": "refine_apply", "run_id": "14a3e38c7569", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "finding_id": "crapi-sqli-001", "namespace": "crapi-demo", "control": "service_policy", "policy": "deny-login-sqli", "lb": "crapi-lab", "passed": true, "attempts": 2, "before_after": {"before": {"exploit_status": 200, "exploit_blocked": false, "legit_ok": true}, "after": {"exploit_status": 403, "exploit_blocked": true, "legit_ok": true}}} -{"ts": "2026-07-27T01:40:29.195575+00:00", "action": "apply_timing", "run_id": "14a3e38c7569", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "control": "service_policy", "finding_id": "crapi-sqli-001", "passed": true, "elapsed_s": 48.0, "attempts": 2, "before_after": {"before": {"exploit_status": 200, "exploit_blocked": false, "legit_ok": true}, "after": {"exploit_status": 403, "exploit_blocked": true, "legit_ok": true}}} -{"ts": "2026-07-27T01:41:21.195575+00:00", "action": "create_api_definition", "run_id": "14a3e38c7569", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "finding_id": "crapi-bola-002", "namespace": "crapi-demo", "name": "crapi-lab-apidef", "swagger": "crapi-lab-swagger"} -{"ts": "2026-07-27T01:41:25.195575+00:00", "action": "apply_api_schema", "run_id": "14a3e38c7569", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "finding_id": "crapi-bola-002", "namespace": "crapi-demo", "apidef": "crapi-lab-apidef", "lb": "crapi-lab", "passed": true, "kept": true, "before_after": {"before": {"exploit_status": 200, "exploit_blocked": false, "legit_ok": true}, "after": {"exploit_status": 403, "exploit_blocked": true, "legit_ok": true}}} -{"ts": "2026-07-27T01:41:29.195575+00:00", "action": "apply_timing", "run_id": "14a3e38c7569", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "control": "api_schema", "finding_id": "crapi-bola-002", "passed": true, "elapsed_s": 33.0, "attempts": 1, "before_after": {"before": {"exploit_status": 200, "exploit_blocked": false, "legit_ok": true}, "after": {"exploit_status": 403, "exploit_blocked": true, "legit_ok": true}}} -{"ts": "2026-07-27T01:42:06.195575+00:00", "action": "apply_waf", "run_id": "14a3e38c7569", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "finding_id": "crapi-mass-003", "namespace": "crapi-demo", "app_firewall": "crapi-lab-waf", "lb": "crapi-lab", "config_enabled": true, "kept": true, "before_after": {"before": {"exploit_status": 200, "exploit_blocked": false, "legit_ok": true}, "after": {"exploit_status": 403, "exploit_blocked": true, "legit_ok": true}}} -{"ts": "2026-07-27T01:42:10.195575+00:00", "action": "apply_timing", "run_id": "14a3e38c7569", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "control": "waf", "finding_id": "crapi-mass-003", "passed": true, "elapsed_s": 21.0, "attempts": 1, "before_after": {"before": {"exploit_status": 200, "exploit_blocked": false, "legit_ok": true}, "after": {"exploit_status": 403, "exploit_blocked": true, "legit_ok": true}}} -{"ts": "2026-07-27T01:42:35.195575+00:00", "action": "apply_rate_limit", "run_id": "14a3e38c7569", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "finding_id": "crapi-bruteforce-004", "namespace": "crapi-demo", "rate": "5/MINUTE", "lb": "crapi-lab", "passed": true, "kept": true, "behavioral": {"sent": 30, "limited": 25, "passed": 5, "codes": {"200": 5, "429": 25}}} -{"ts": "2026-07-27T01:42:39.195575+00:00", "action": "apply_timing", "run_id": "14a3e38c7569", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "control": "rate_limit", "finding_id": "crapi-bruteforce-004", "passed": true, "elapsed_s": 27.0, "attempts": 1} -{"ts": "2026-07-27T01:43:10.195575+00:00", "action": "apply_data_guard", "run_id": "14a3e38c7569", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "finding_id": "crapi-tokenleak-006", "namespace": "crapi-demo", "app_firewall": "crapi-lab-waf", "lb": "crapi-lab", "enabled": true, "kept": true} -{"ts": "2026-07-27T01:43:14.195575+00:00", "action": "apply_timing", "run_id": "14a3e38c7569", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "control": "waf_data_guard", "finding_id": "crapi-tokenleak-006", "passed": true, "elapsed_s": 19.0, "attempts": 1} -{"ts": "2026-07-27T01:43:37.195575+00:00", "action": "open_pr", "run_id": "14a3e38c7569", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "finding_id": "crapi-sqli-001", "finding": "crapi-sqli-001", "repo": "acme/crapi", "url": "https://github.com/acme/crapi/pull/311", "number": 311} -{"ts": "2026-07-27T01:43:41.195575+00:00", "action": "retire", "run_id": "14a3e38c7569", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "finding_id": "crapi-sqli-001", "namespace": "crapi-demo", "control": "service_policy", "lb": "crapi-lab", "forced": false} +{"ts": "2026-07-27T15:42:54.260296+00:00", "action": "refine_apply", "run_id": "f0c2bd6bc96c", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "finding_id": "crapi-sqli-001", "namespace": "crapi-demo", "control": "service_policy", "policy": "deny-login-sqli", "lb": "crapi-lab", "passed": true, "attempts": 2, "before_after": {"before": {"exploit_status": 200, "exploit_blocked": false, "legit_ok": true}, "after": {"exploit_status": 403, "exploit_blocked": true, "legit_ok": true}}} +{"ts": "2026-07-27T15:42:58.260296+00:00", "action": "apply_timing", "run_id": "f0c2bd6bc96c", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "control": "service_policy", "finding_id": "crapi-sqli-001", "passed": true, "elapsed_s": 48.0, "attempts": 2, "before_after": {"before": {"exploit_status": 200, "exploit_blocked": false, "legit_ok": true}, "after": {"exploit_status": 403, "exploit_blocked": true, "legit_ok": true}}} +{"ts": "2026-07-27T15:43:50.260296+00:00", "action": "create_api_definition", "run_id": "f0c2bd6bc96c", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "finding_id": "crapi-bola-002", "namespace": "crapi-demo", "name": "crapi-lab-apidef", "swagger": "crapi-lab-swagger"} +{"ts": "2026-07-27T15:43:54.260296+00:00", "action": "apply_api_schema", "run_id": "f0c2bd6bc96c", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "finding_id": "crapi-bola-002", "namespace": "crapi-demo", "apidef": "crapi-lab-apidef", "lb": "crapi-lab", "passed": true, "kept": true, "before_after": {"before": {"exploit_status": 200, "exploit_blocked": false, "legit_ok": true}, "after": {"exploit_status": 403, "exploit_blocked": true, "legit_ok": true}}} +{"ts": "2026-07-27T15:43:58.260296+00:00", "action": "apply_timing", "run_id": "f0c2bd6bc96c", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "control": "api_schema", "finding_id": "crapi-bola-002", "passed": true, "elapsed_s": 33.0, "attempts": 1, "before_after": {"before": {"exploit_status": 200, "exploit_blocked": false, "legit_ok": true}, "after": {"exploit_status": 403, "exploit_blocked": true, "legit_ok": true}}} +{"ts": "2026-07-27T15:44:35.260296+00:00", "action": "apply_waf", "run_id": "f0c2bd6bc96c", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "finding_id": "crapi-mass-003", "namespace": "crapi-demo", "app_firewall": "crapi-lab-waf", "lb": "crapi-lab", "config_enabled": true, "kept": true, "before_after": {"before": {"exploit_status": 200, "exploit_blocked": false, "legit_ok": true}, "after": {"exploit_status": 403, "exploit_blocked": true, "legit_ok": true}}} +{"ts": "2026-07-27T15:44:39.260296+00:00", "action": "apply_timing", "run_id": "f0c2bd6bc96c", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "control": "waf", "finding_id": "crapi-mass-003", "passed": true, "elapsed_s": 21.0, "attempts": 1, "before_after": {"before": {"exploit_status": 200, "exploit_blocked": false, "legit_ok": true}, "after": {"exploit_status": 403, "exploit_blocked": true, "legit_ok": true}}} +{"ts": "2026-07-27T15:45:04.260296+00:00", "action": "apply_rate_limit", "run_id": "f0c2bd6bc96c", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "finding_id": "crapi-bruteforce-004", "namespace": "crapi-demo", "rate": "5/MINUTE", "lb": "crapi-lab", "passed": true, "kept": true, "behavioral": {"sent": 30, "limited": 25, "passed": 5, "codes": {"200": 5, "429": 25}}} +{"ts": "2026-07-27T15:45:08.260296+00:00", "action": "apply_timing", "run_id": "f0c2bd6bc96c", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "control": "rate_limit", "finding_id": "crapi-bruteforce-004", "passed": true, "elapsed_s": 27.0, "attempts": 1} +{"ts": "2026-07-27T15:45:39.260296+00:00", "action": "apply_data_guard", "run_id": "f0c2bd6bc96c", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "finding_id": "crapi-tokenleak-006", "namespace": "crapi-demo", "app_firewall": "crapi-lab-waf", "lb": "crapi-lab", "enabled": true, "kept": true} +{"ts": "2026-07-27T15:45:43.260296+00:00", "action": "apply_timing", "run_id": "f0c2bd6bc96c", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "control": "waf_data_guard", "finding_id": "crapi-tokenleak-006", "passed": true, "elapsed_s": 19.0, "attempts": 1} +{"ts": "2026-07-27T15:46:06.260296+00:00", "action": "open_pr", "run_id": "f0c2bd6bc96c", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "finding_id": "crapi-sqli-001", "finding": "crapi-sqli-001", "repo": "acme/crapi", "url": "https://github.com/acme/crapi/pull/311", "number": 311} +{"ts": "2026-07-27T15:46:10.260296+00:00", "action": "retire", "run_id": "f0c2bd6bc96c", "actor": "security-oncall", "host": "vpcopilot-demo", "tool_version": "0.1.0", "finding_id": "crapi-sqli-001", "namespace": "crapi-demo", "control": "service_policy", "lb": "crapi-lab", "forced": false} diff --git a/demo/out/report.html b/demo/out/report.html index 68f7305..71b4741 100644 --- a/demo/out/report.html +++ b/demo/out/report.html @@ -57,7 +57,7 @@ .model .a{color:var(--grey)}.model .m{font-family:ui-monospace,Menlo,monospace;color:var(--f5)}

virtual-patch·copilot — scan report

-
target: crapi-lab · generated 2026-07-27 01:49 UTC
+
target: crapi-lab · generated 2026-07-27 15:51 UTC
6exploitable vulns
→
5mitigated live by XC
29.6stime to mitigate · 72,973× faster
vs
25 daysnormal change control
6code-fix PRs (the cure)

Run summary

9candidates
6verified
5band-aided
1code-cure only
5XC policies
6code-fix PRs
@@ -66,7 +66,7 @@

Model independence each agent's model is set per-agent in

Pipeline metrics

19.2stotal time
6.4sdiscover
4.1sverify
8.7ssynthesize
67%verify confirm-rate
0.86avg confidence
1dupe band-aids collapsed

verify: 9 candidates → 6 verified, 2 refuted, 1 dropped < 0.5 confidence

Findings & band-aid coverage

criticalSQL injection in logincrapi-sqli-001sqliservices/identity/login.js:42
service_policy · full✓ code fix drafted
Details

Description: The email field is concatenated straight into the auth query.

Exploit: email=" OR 1=1 -- lets any password through and dumps the users table.

Code cure: Fix sql injection in login

highBOLA on vehicle locationcrapi-bola-002broken_object_authzservices/identity/vehicle.js:88
api_schema · full✓ code fix drafted
Details

Description: Any authenticated user can read another user's vehicle GPS by id.

Exploit: Swap {id} to another user's vehicle uuid; server returns their live location.

Code cure: Fix bola on vehicle location

highMass assignment on profilecrapi-mass-003mass_assignmentservices/identity/dashboard.js:61
waf · partial✓ code fix drafted
Details

Description: The update handler binds the whole body, so `role` and `credit` are writable.

Exploit: POST {"role":"admin","available_credit":9999} — privilege + balance escalation.

Code cure: Fix mass assignment on profile

highJWT + card data in responsecrapi-tokenleak-006sensitive_dataservices/workshop/mechanic.js:120
waf_data_guard · partial✓ code fix drafted
Details

Description: The receipt payload echoes the full PAN and a signed service token.

Exploit: GET a receipt; response body contains the 16-digit card number in cleartext.

Code cure: Fix jwt + card data in response

mediumNo rate limit on OTP verifycrapi-bruteforce-004rate_abuseservices/identity/otp.js:30
rate_limit · full✓ code fix drafted
Details

Description: The 4-digit OTP endpoint has no throttle — brute-forceable in minutes.

Exploit: Fire all 10k OTPs; no lockout, no delay.

Code cure: Fix no rate limit on otp verify

mediumUsername enumeration on signupcrapi-userenum-005broken_authservices/identity/signup.js:25
no band-aid — code cure only✓ code fix drafted
Residual risk: no positive-security band-aid fits; ships as code-only.
Details

Description: Distinct errors for taken vs free emails leak which accounts exist.

Exploit: Diff the 'already registered' vs 'ok' responses to enumerate users.

Code cure: Fix username enumeration on signup

Generated XC band-aid policies

api_schema crapi-lab-apidef
rate_limit otp-throttle
service_policy deny-login-sqli
waf crapi-lab-waf
waf_data_guard mask-pan
-

Band-aid impact exploit before → after (live validation)

controlpolicyexploit beforeexploit afterlegitresultwhen
service_policy self-healed ×2deny-login-sqli200 allowed403 blockedokPASS2026-07-27T01:40:25
apply_timing self-healed ×2200 allowed403 blockedokPASS2026-07-27T01:40:29
api_schemacrapi-lab-apidef200 allowed403 blockedokPASS2026-07-27T01:41:25
apply_timing200 allowed403 blockedokPASS2026-07-27T01:41:29
wafcrapi-lab-waf200 allowed403 blockedokfail2026-07-27T01:42:06
apply_timing200 allowed403 blockedokPASS2026-07-27T01:42:10
rate_limit5/MINUTEburst 30 allowed25/30 rate-limited (429)—PASS2026-07-27T01:42:35
+

Band-aid impact exploit before → after (live validation)

controlpolicyexploit beforeexploit afterlegitresultwhen
service_policy self-healed ×2deny-login-sqli200 allowed403 blockedokPASS2026-07-27T15:42:54
apply_timing self-healed ×2200 allowed403 blockedokPASS2026-07-27T15:42:58
api_schemacrapi-lab-apidef200 allowed403 blockedokPASS2026-07-27T15:43:54
apply_timing200 allowed403 blockedokPASS2026-07-27T15:43:58
wafcrapi-lab-waf200 allowed403 blockedokfail2026-07-27T15:44:35
apply_timing200 allowed403 blockedokPASS2026-07-27T15:44:39
rate_limit5/MINUTEburst 30 allowed25/30 rate-limited (429)—PASS2026-07-27T15:45:04

Blast radius what each band-aid would block in recorded traffic — 500 of 500 recorded request(s) replayed through crapi-lab · window 2026-07-20T09:00:00Z..2026-07-20T10:00:00Z

policyevaluatedwould blockrateverdicttop blocked paths
deny-login-sqli50030.6%within threshold/identity/api/auth/login ×3

Remediation ledger found → mitigated → remediated → retired

findingstateband-aidcode cure
crapi-sqli-001retired{'control': 'service_policy', 'policy_name': 'deny-login-sqli', 'lb': 'crapi-lab'}{'pr_url': 'https://github.com/acme/crapi/pull/311', 'pr_number': 311}
crapi-bola-002remediated{'control': 'api_schema', 'policy_name': 'crapi-lab-apidef', 'lb': 'crapi-lab'}{'pr_url': 'https://github.com/acme/crapi/pull/312', 'pr_number': 312}
crapi-mass-003mitigated{'control': 'waf', 'policy_name': 'crapi-lab-waf', 'lb': 'crapi-lab'}—
crapi-bruteforce-004mitigated{'control': 'rate_limit', 'policy_name': 'otp-throttle', 'lb': 'crapi-lab'}—
crapi-tokenleak-006remediated{'control': 'waf_data_guard', 'policy_name': 'mask-pan', 'lb': 'crapi-lab'}{'pr_url': 'https://github.com/acme/crapi/pull/313', 'pr_number': 313}
crapi-userenum-005found——

virtual-patch-copilot 0.1.0 · band-aids are temporary — every finding also gets a code-fix PR
diff --git a/demo/out/run.json b/demo/out/run.json index 0936b2b..abe5f53 100644 --- a/demo/out/run.json +++ b/demo/out/run.json @@ -1,6 +1,6 @@ { - "run_id": "14a3e38c7569", - "created": "2026-07-27T01:49:25.186311+00:00", + "run_id": "f0c2bd6bc96c", + "created": "2026-07-27T15:51:54.251818+00:00", "repo": "/src/crapi", "config_path": "config/agents.yaml", "models": { diff --git a/demo/out/simulation.json b/demo/out/simulation.json new file mode 100644 index 0000000..5c45ea1 --- /dev/null +++ b/demo/out/simulation.json @@ -0,0 +1,60 @@ +{ + "ts": "2026-07-27T15:51:54.261672+00:00", + "lb": "crapi-lab", + "source": "xc:crapi-lab", + "records": 500, + "records_replayed": 500, + "window": "2026-07-20T09:00:00Z..2026-07-20T10:00:00Z", + "redacted": { + "authorization": 500, + "cookie": 431, + "password": 12 + }, + "bodies_available": true, + "policies": [ + { + "policy_name": "deny-login-sqli", + "control": "service_policy", + "finding_id": "crapi-sqli-001", + "evaluated": 500, + "would_block": 3, + "block_rate": 0.006, + "threshold": 0.01, + "blocked_promotion": false, + "reason": "", + "errored": 0, + "enforcement_confirmed": true, + "samples": [ + { + "method": "POST", + "path": "/identity/api/auth/login", + "source": "xc", + "json_body": null, + "user_agent": "python-requests/2.31", + "ts": "2026-07-20T09:14:02Z", + "status": 200 + } + ], + "top_paths": [ + [ + "/identity/api/auth/login", + 3 + ] + ], + "top_user_agents": [ + [ + "python-requests/2.31", + 2 + ], + [ + "sqlmap/1.7", + 1 + ] + ] + } + ], + "caveats": [ + "A would-block count describes THIS sample only \u2014 a quiet window makes any policy look safe. The window and record count are stated above; read them together with the rate.", + "Replay runs one policy at a time on a spare LB. Two candidates share FIRST_MATCH ordering, so a batched result would not describe either band-aid alone." + ] +} \ No newline at end of file diff --git a/docs/AUDIT.md b/docs/AUDIT.md index ff6b5b4..3f1dfa2 100644 --- a/docs/AUDIT.md +++ b/docs/AUDIT.md @@ -359,7 +359,7 @@ To re-derive `audit.csv` yourself and confirm the normalization added nothing, r ## 7. How to get one -### Console — ⑤ Retire +### Console — ⑥ Retire ```sh vpcopilot console # http://127.0.0.1:8787 diff --git a/docs/DEMO.md b/docs/DEMO.md index 26ccf6c..c269876 100644 --- a/docs/DEMO.md +++ b/docs/DEMO.md @@ -35,10 +35,10 @@ Walk the steps top to bottom — the whole arc is already in the data: exploit actually returned 403 — and rate-limit shows the behavioral proof (25/30 requests 429'd). **Download** grabs a stamped copy. It is rebuilt from the current out dir every time you open it, so it is never a stale file. -4. **⑤ Retire → ledger.** The four-state track: `found → mitigated → remediated → retired`. +4. **⑥ Retire → ledger.** The four-state track: `found → mitigated → remediated → retired`. `crapi-sqli-001` is walked all the way to **retired** — its cure PR merged, so the band-aid was detached. -5. **⑤ Retire → audit trail.** *Every change made to a load balancer*, one row each: when (UTC) · +5. **⑥ Retire → audit trail.** *Every change made to a load balancer*, one row each: when (UTC) · action · justified by (the finding + severity) · control (+ the XC object) · load balancer (+ namespace) · outcome (with the `200 allowed → 403 blocked` proof and a self-heal ×N badge) · by. Filter it, expand `▸` for the raw JSON, then **Export evidence bundle (.zip)** — the @@ -71,15 +71,15 @@ vpcopilot console # http://127.0.0.1:8787 scroll up mid-scan to re-read the discover output and it stays put; a **↓ follow** chip and a line counter appear until you scroll back to the bottom. Long scans no longer push the page down. 2. **② Review** the findings, and hit **Open HTML report ↗** if someone wants the artifact now. -3. **③ Mitigate.** With `dry-run` OFF and `keep live` ON (**Run settings** — the collapsible bar at the top of the Mitigate step), click +3. **④ Mitigate.** With `dry-run` OFF and `keep live` ON (**Run settings** — the collapsible bar at the top of the Mitigate step), click **Mitigate service_policy ▶** on a finding. The refiner streams in the row: attach → validate → (refine → retry)* → **before 200 through → after 403 BLOCKED · legit ok**, with a *self-healed in N attempts* badge if it took more than one try. It never claims success unless the live exploit is actually blocked. 4. **XC security dashboard ↗** (hero band) — jump to the native WAF/API-Security telemetry to show the block landing in XC. -5. **④ Cure → Open PR** on the same finding to draft the real code fix against your repo. -6. **⑤ Retire** once the cure merges — the band-aid is detached and the finding goes `retired`. The +5. **⑤ Cure → Open PR** on the same finding to draft the real code fix against your repo. +6. **⑥ Retire** once the cure merges — the band-aid is detached and the finding goes `retired`. The loop is closed. Below the ledger, the **audit trail** now has a row per live change: which finding justified it, which LB and namespace it touched, whether it stuck, and who ran it (`VPCOPILOT_ACTOR`, else the OS user). **Export evidence bundle (.zip)** hands that to whoever @@ -118,13 +118,14 @@ story on their own: |---|---| | Scan — the target form and its scrollable run log | [`1-scan.png`](images/1-scan.png) | | Review — hero band + findings + the HTML-report buttons | [`2-review.png`](images/2-review.png) | -| Mitigate — per-finding live apply | [`3-mitigate.png`](images/3-mitigate.png) | -| Retire — four-state ledger (`crapi-sqli-001` at *retired*) + the audit trail | [`5-retire.png`](images/5-retire.png) | +| Simulate — blast radius of each candidate before it is applied | [`3-simulate.png`](images/3-simulate.png) | +| Mitigate — per-finding live apply | [`4-mitigate.png`](images/4-mitigate.png) | +| Retire — four-state ledger (`crapi-sqli-001` at *retired*) + the audit trail | [`6-retire.png`](images/6-retire.png) | | The shareable HTML report (self-heal ×2 + rate-limit proof) | [`report.png`](images/report.png) | To regenerate them: rebuild the dataset with `python3 demo/build_demo_out.py`, run `VPCOPILOT_OUT=demo/out vpcopilot console`, then capture the `#scan`, `#review`, `#mitigate` and -`#retire` steps plus `demo/out/report.html` at 1200px wide / 2× device pixel ratio. +`#simulate`, `#mitigate` and `#retire` steps plus `demo/out/report.html` at 1200px wide / 2× device pixel ratio. Point the console at a **credential-free** `.env` when you do (`VPCOPILOT_ENV=…`): with XC creds loaded, the hero band renders a deep link carrying your tenant hostname and namespace, and that diff --git a/docs/PROJECT_STATE.md b/docs/PROJECT_STATE.md index 3bfb14b..957a1d6 100644 --- a/docs/PROJECT_STATE.md +++ b/docs/PROJECT_STATE.md @@ -21,7 +21,7 @@ real exploit (self-heal until it blocks, or honestly "unfixable") → open the c - **Local open-source model wired** — `config/agents.dgx.yaml` runs every agent on a local Ollama/vLLM server (structured output validated live). The third leg of the benchmark; see below. - **The console is now a full benchmark-driving surface** — model dropdown, data-backed pickers - (load balancer, scan target, output dir, PR repo), **Mitigate ALL**, and a **⑥ Benchmark** + (load balancer, scan target, output dir, PR repo), **Mitigate ALL**, and a **⑦ Benchmark** build+compare step. The whole three-way runs without leaving the UI. - **Audit capture + evidence export (F3) landed** — the answer to "why was this LB changed, by whom, in which tenant?". Every LB/object-mutating record now carries `finding_id` + `namespace` @@ -31,7 +31,7 @@ real exploit (self-heal until it blocks, or honestly "unfixable") → open the c models + caps + counts) and `export.py` (normalized events → `audit.csv`/`audit-events.json` + the raw artifacts + a SHA-256'd `manifest.json`, stdlib-only). Surfaces: `vpcopilot export [--all]`, `GET /api/audit-events` · `/api/audit-export?scope=run|all` · `/api/runs`, and an - **Audit trail** card on ⑤ Retire that shows the trail before you export it. New env var + **Audit trail** card on ⑥ Retire that shows the trail before you export it. New env var **`VPCOPILOT_ACTOR`** (defaults to the OS user). - **Console log windows are complete and scrollable** — `/api/scan` and `/api/action` now return the FULL transcript plus `log_total` and take `?since=` for the new tail only (they used to @@ -76,7 +76,7 @@ real exploit (self-heal until it blocks, or honestly "unfixable") → open the c `retire`, `open_pr`, `create_*` and config-only applies). Identity is stamped inside `audit.record`, so add a new mutating path and it is attributable for free — and a caller cannot override it. - **Benchmark harness** — `bench_model.py` + `vpcopilot bench-model` / `bench-compare`, **also in - the console's ⑥ Benchmark step** (`POST /api/bench-model` build + `GET /api/benchmarks` compare + the console's ⑦ Benchmark step** (`POST /api/bench-model` build + `GET /api/benchmarks` compare table — same `benchmarks/*.json` as the CLI). Reads a run's findings/policies + the audit log's `apply_timing` records → findings, policies-by-control, and **live policy quality** (blocked / applied-behavioral / failed / self-healed). @@ -130,9 +130,9 @@ Now fully UI-drivable (headless CLI still works — add `--no-code-fixes` to `vp 3. **① Scan** (Target repo + Output dir are comboboxes — pick the app) → **② Review**. 4. **Run settings**: pick the app's **load balancer** (the picker lists the namespace's LBs and **auto-fills the validate URL** from its domain), **dry-run OFF**, **keep OFF** (validate then - roll back — LB stays clean, quality still recorded) → **③ Mitigate ALL** (one click; applies + roll back — LB stays clean, quality still recorded) → **④ Mitigate ALL** (one click; applies every band-aid sequentially). -5. **⑥ Benchmark** → tag the run (e.g. `dgx-vampi`) → **Build benchmark** → the compare table +5. **⑦ Benchmark** → tag the run (e.g. `dgx-vampi`) → **Build benchmark** → the compare table updates across all runs. (CLI equivalents: `bench-model --tag … --out … --config …`, then `bench-compare benchmarks/*.json`.) diff --git a/docs/TRY_IT.md b/docs/TRY_IT.md index eaa1511..7af75c5 100644 --- a/docs/TRY_IT.md +++ b/docs/TRY_IT.md @@ -106,12 +106,12 @@ vpcopilot console ``` Set the **Load balancer** and **Validate URL** in *Run settings* to your lab LB + URL, then: -- **③ Mitigate** — click *Mitigate* on a finding. With dry-run **off**, the copilot attaches the +- **④ Mitigate** — click *Mitigate* on a finding. With dry-run **off**, the copilot attaches the band-aid, fires the finding's real exploit, refines until it's actually blocked, and shows `before 200 → after 403 BLOCKED`. Leave *keep live* off to auto-roll-back after validating (a safe smoke), or on to keep it enforcing. -- **④ Cure** — set a *PR repo* you can push to, then *Open PR* to draft the code fix. -- **⑤ Retire** — once the cure merges, *Retire* detaches the temporary XC control. +- **⑤ Cure** — set a *PR repo* you can push to, then *Open PR* to draft the code fix. +- **⑥ Retire** — once the cure merges, *Retire* detaches the temporary XC control. The equivalent CLI is in [USAGE.md](USAGE.md) (`apply --from-scan …`, `pr …`, `retire …`). diff --git a/docs/USAGE.md b/docs/USAGE.md index 1acd59d..862c9a1 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -71,6 +71,8 @@ Uses the full corrected file from `remediate` (no fragile diff apply). Token fro ## 6. Track & audit ```sh +vpcopilot simulate --out out --logs traffic.har # what each band-aid WOULD block +vpcopilot simulate --out out --from-tenant --source-lb --since 6h vpcopilot ledger # found -> mitigated -> remediated -> retired (per finding) vpcopilot audit # append-only log of every applied / rolled-back change vpcopilot export [--out DIR] [--output PATH] # evidence bundle (.zip) for one run @@ -84,7 +86,7 @@ vpcopilot retire --all # retire every mitigated finding whose cure PR an explicit `caveats` list), `audit.csv` + `audit-events.json` (one normalized row per change, joined to the finding that justified it), the raw `audit.log` verbatim, `run.json`, the ledger and scan artifacts, `policies/*` (the exact XC configs pushed), `snapshots/*` (pre-change LB state), and -`report.html`. It is the same bundle the console's ⑤ Retire step downloads, and it is read-only — +`report.html`. It is the same bundle the console's ⑥ Retire step downloads, and it is read-only — nothing here touches XC or GitHub. Dry runs are not in it: nothing changed, so nothing is logged. The bundle is evidence for a human @@ -107,10 +109,11 @@ header carries a live model switcher, and each step is deep-linkable (`#mitigate |---|---| | **① Scan** | point at a repo and run the pipeline — read-only, no XC/GitHub writes. Auto-advances to Review when it finishes | | **② Review** | verified findings + the recommended band-aid; click a row for exploit / code / generated policy. **Open HTML report ↗** + **Download** | -| **③ Mitigate** | apply each band-aid (or **Mitigate ALL**, one at a time, continuing past failures) and watch `before → after` stream, with a *self-healed in N attempts* badge | -| **④ Cure** | open the code-fix PR per finding, or all of them | -| **⑤ Retire** | the four-state ledger track, plus the **Audit trail** table and **Export evidence bundle (.zip)** / **All runs** | -| **⑥ Benchmark** | build a model-tagged report from this run, then compare models side by side per target app | +| **③ Simulate** | replay a recorded sample against each candidate through a **spare** LB and report what it would block; over-threshold policies warn at the gate | +| **④ Mitigate** | apply each band-aid (or **Mitigate ALL**, one at a time, continuing past failures) and watch `before → after` stream, with a *self-healed in N attempts* badge | +| **⑤ Cure** | open the code-fix PR per finding, or all of them | +| **⑥ Retire** | the four-state ledger track, plus the **Audit trail** table and **Export evidence bundle (.zip)** / **All runs** | +| **⑦ Benchmark** | build a model-tagged report from this run, then compare models side by side per target app | | **⚙ Setup** | credentials (writes `.env`), XC status, the per-agent model wiring, and the report buttons | **Run settings** — the collapsible bar shown on the action steps (**Mitigate / Cure / Retire**): @@ -118,14 +121,14 @@ LB · validate URL · PR repo · base · path prefix, plus **dry-run** (on by de attempts, **keep live**, and **allow protected LB**. Its summary line spells out the mode you're about to run in — `dry-run · rollback · LB=… · refine×3`. -**Log windows.** ① Scan and ③ Mitigate's per-finding job log hold the *whole* transcript in a +**Log windows.** ① Scan and ④ Mitigate's per-finding job log hold the *whole* transcript in a scrollable box, not the last N lines. The endpoints serve the full log and the page appends only the new tail, so scroll position and text selection survive each poll — you can read back through a long run while it's still going. Both stick to the bottom only while you're already at the bottom; on ① Scan, scrolling up also reveals a **↓ follow** chip and a line count (the Mitigate job log has neither — it's a small box inside a table row). -**Audit trail (⑤ Retire).** One row per change made to a load balancer — when (UTC) · action · +**Audit trail (⑥ Retire).** One row per change made to a load balancer — when (UTC) · action · justified by (the finding, its id and severity) · control (+ the XC object) · load balancer (+ namespace) · outcome (with a self-heal ×N badge and the `200 allowed → 403 blocked` proof) · by (actor). Filter it, expand `▸` for the raw JSON, then **Export evidence bundle (.zip)** for this run diff --git a/docs/images/1-scan.png b/docs/images/1-scan.png index 8896f00..fb78d0f 100644 Binary files a/docs/images/1-scan.png and b/docs/images/1-scan.png differ diff --git a/docs/images/2-review.png b/docs/images/2-review.png index e17ddd3..c9d57e2 100644 Binary files a/docs/images/2-review.png and b/docs/images/2-review.png differ diff --git a/docs/images/3-mitigate.png b/docs/images/3-mitigate.png deleted file mode 100644 index 33b0447..0000000 Binary files a/docs/images/3-mitigate.png and /dev/null differ diff --git a/docs/images/3-simulate.png b/docs/images/3-simulate.png new file mode 100644 index 0000000..462c04a Binary files /dev/null and b/docs/images/3-simulate.png differ diff --git a/docs/images/4-mitigate.png b/docs/images/4-mitigate.png new file mode 100644 index 0000000..d6619d5 Binary files /dev/null and b/docs/images/4-mitigate.png differ diff --git a/docs/images/5-retire.png b/docs/images/5-retire.png deleted file mode 100644 index 5d4abb9..0000000 Binary files a/docs/images/5-retire.png and /dev/null differ diff --git a/docs/images/6-retire.png b/docs/images/6-retire.png new file mode 100644 index 0000000..e94acb5 Binary files /dev/null and b/docs/images/6-retire.png differ diff --git a/docs/images/report.png b/docs/images/report.png index 704fe03..ff01666 100644 Binary files a/docs/images/report.png and b/docs/images/report.png differ diff --git a/src/vpcopilot/cli.py b/src/vpcopilot/cli.py index 4f42b28..ec77c95 100644 --- a/src/vpcopilot/cli.py +++ b/src/vpcopilot/cli.py @@ -434,6 +434,86 @@ def audit(out: str = typer.Option("out", help="output directory")): rprint(t) +@app.command() +def simulate( + out: str = typer.Option("out", help="run directory holding the generated policies"), + logs: str = typer.Option(None, "--logs", help="traffic sample: .har / .json (HAR) or .jsonl"), + from_tenant: bool = typer.Option(False, "--from-tenant", help="read observed requests from XC access logs"), + lb: str = typer.Option("vpcopilot-lab", help="spare LB to replay through (never a protected one)"), + url: str = typer.Option("https://lab.banknimbus.com", help="base URL of that LB"), + source_lb: str = typer.Option(None, "--source-lb", help="with --from-tenant: the LB whose traffic to read"), + since: str = typer.Option("1h", "--since", help="with --from-tenant: window back from now, e.g. 30m / 6h"), + limit: int = typer.Option(500, help="max records to pull from the tenant"), + max_records: int = typer.Option(200, help="max records to actually replay (each is one live request)"), + threshold: float = typer.Option(None, help="would-block rate that flags a policy (default 0.01)"), + policy: str = typer.Option(None, "--policy", help="simulate only this policy"), +): + """Replay a recorded traffic sample against each generated band-aid and report what it WOULD + block, before anything reaches the gate. Read-only against the sample; the spare LB is + snapshotted and restored.""" + import os + from .simulate import DEFAULT_THRESHOLD, candidates_from_out, simulate_policies, write_result + + cands = candidates_from_out(out, policy) + if not cands: + rprint(f"[yellow]no service_policy artifacts in {out} — nothing to simulate[/yellow]") + raise typer.Exit(code=1) + + records, redacted, src, window = _load_traffic(logs, from_tenant, source_lb or lb, since, limit) + if not records: + rprint("[yellow]no records ingested — supply --logs or --from-tenant[/yellow]") + raise typer.Exit(code=1) + rprint(f"[dim]{len(records)} record(s) from {src}" + + (f"; redacted {sum(v for k, v in redacted.items() if not k.startswith('_'))} value(s)" if redacted else "") + + "[/dim]") + + thr = threshold if threshold is not None else float(os.environ.get("VPCOPILOT_SIM_THRESHOLD", DEFAULT_THRESHOLD)) + res = simulate_policies(cands, records, lb=lb, url=url, out_dir=out, threshold=thr, + max_records=max_records, source=src, window=window, redacted=redacted, + log=lambda m: rprint(f"[dim]{m}[/dim]")) + path = write_result(out, res) + t = Table(title="blast radius") + for c in ["policy", "evaluated", "would block", "rate", "verdict"]: + t.add_column(c) + for p in res.policies: + verdict = ("[red]over threshold[/red]" if p.blocked_promotion + else ("[yellow]error[/yellow]" if p.error else "[green]ok[/green]")) + t.add_row(p.policy_name, str(p.evaluated), str(p.would_block), f"{p.block_rate:.1%}", verdict) + rprint(t) + rprint(f"wrote [bold]{path}[/bold]") + + +def _load_traffic(logs, from_tenant, source_lb, since, limit): + """Traffic comes from a file, the tenant, or both — the two sources are additive on purpose: + a HAR carries bodies the access logs cannot.""" + import datetime as _dt + + from . import traffic + records, redacted, srcs = [], {}, [] + window = "" + if logs: + recs, red = traffic.load(logs) + records += recs + redacted.update(red) + srcs.append(f"file:{logs}") + if from_tenant: + from .xc import XC + n = int("".join(ch for ch in since if ch.isdigit()) or 1) + unit = since.strip()[-1].lower() + delta = _dt.timedelta(**{{"m": "minutes", "h": "hours", "d": "days"}.get(unit, "hours"): n}) + now = _dt.datetime.now(_dt.timezone.utc) + start, end = now - delta, now + rows = XC().access_logs(start=start.strftime("%Y-%m-%dT%H:%M:%SZ"), + end=end.strftime("%Y-%m-%dT%H:%M:%SZ"), limit=limit, lb=source_lb) + recs, red = traffic.from_xc_logs(rows, lb=source_lb) + records += recs + for k, v in red.items(): + redacted[k] = redacted.get(k, 0) + v + srcs.append(f"xc:{source_lb}") + window = f"{start:%Y-%m-%dT%H:%M:%SZ}..{end:%Y-%m-%dT%H:%M:%SZ}" + return records, redacted, " + ".join(srcs), window + + @app.command() def export( out: str = typer.Option("out", help="run directory to export"), diff --git a/src/vpcopilot/console/app.py b/src/vpcopilot/console/app.py index 00d9e6b..ae6b74b 100644 --- a/src/vpcopilot/console/app.py +++ b/src/vpcopilot/console/app.py @@ -173,6 +173,68 @@ def audit_events(): return {"out": str(OUT), "events": build_audit_events(str(OUT))} +@app.get("/api/simulate") +def simulation(): + """G2: the last blast-radius result for this run — what each candidate band-aid WOULD block, + measured by replaying a recorded sample through a spare LB. Empty until a simulation runs; + the Simulate step reads this to render before anything reaches the gate.""" + from ..simulate import load_result + return {"out": str(OUT), "simulation": load_result(str(OUT))} + + +class SimReq(BaseModel): + logs: str | None = None # HAR / JSONL path + from_tenant: bool = False # read observed requests from XC access logs + lb: str = "vpcopilot-lab" # the spare LB to replay through + url: str = "https://lab.banknimbus.com" + source_lb: str | None = None # whose traffic to read, when from_tenant + since: str = "1h" + limit: int = 500 + max_records: int = 200 + threshold: float | None = None + policy: str | None = None + + +@app.post("/api/simulate") +def run_simulation(body: SimReq): + """Run a replay in the background, streaming through the same job contract as apply — the + console polls /api/action?job=… for the log.""" + import uuid + load_dotenv(ENV_PATH, override=True) + job_id = uuid.uuid4().hex[:8] + _jobs[job_id] = {"state": "running", "log": [], "result": None, "error": None, + "control": "simulate", "finding_id": None} + threading.Thread(target=_run_simulation, args=(job_id, body), daemon=True).start() + return {"job": job_id, "state": "running"} + + +def _run_simulation(job_id: str, body: SimReq): + job = _jobs[job_id] + log = lambda m: _append(job["log"], m) # noqa: E731 + try: + import os + + from ..cli import _load_traffic + from ..simulate import DEFAULT_THRESHOLD, candidates_from_out, simulate_policies, write_result + cands = candidates_from_out(str(OUT), body.policy) + if not cands: + raise RuntimeError(f"no service_policy artifacts in {OUT} — run a scan first") + records, redacted, src, window = _load_traffic( + body.logs, body.from_tenant, body.source_lb or body.lb, body.since, body.limit) + if not records: + raise RuntimeError("no records ingested — give a traffic file or enable from-tenant") + log(f"{len(records)} record(s) from {src}") + thr = body.threshold if body.threshold is not None else float( + os.environ.get("VPCOPILOT_SIM_THRESHOLD", DEFAULT_THRESHOLD)) + res = simulate_policies(cands, records, lb=body.lb, url=body.url, out_dir=str(OUT), + threshold=thr, max_records=body.max_records, source=src, + window=window, redacted=redacted, log=log) + write_result(str(OUT), res) + job.update(state="done", result=res.model_dump()) + except Exception as e: # noqa: BLE001 + job.update(state="error", error=str(e)) + + @app.get("/api/runs") def runs(): """Run dirs on disk that have something to export — backs the Retire step's 'all runs' bundle.""" @@ -538,6 +600,7 @@ class ActionReq(BaseModel): refine: bool = True refine_attempts: int | None = None allow_protected_lb: bool = False + allow_overbroad: bool = False # G2: apply anyway when simulation flagged the policy as too broad _jobs: dict[str, dict] = {} # job_id -> {state, log, result, error, control, finding_id} @@ -552,6 +615,19 @@ def _dispatch_action(body: ActionReq, log): c, kw = body.control, dict(finding_id=body.finding_id, dry_run=body.dry_run, keep=body.keep, allow_protected=body.allow_protected_lb, out_dir=str(OUT), log=log) if c == "service_policy": + # G2 gate: a simulated policy found too broad WARNS and requires an explicit override. + # Silent when nothing was simulated — G2 adds a check, never a prerequisite. + from ..simulate import promotion_block + over = promotion_block(str(OUT), body.policy_name) if body.policy_name else None + if over and not body.dry_run: + if not body.allow_overbroad: + raise HTTPException(409, f"simulation says this policy {over.get('reason')}. " + "Re-run with 'allow overbroad' to apply it anyway.") + log(f"⚠ overbroad override: {over.get('reason')}") + from ..audit import record as _audit + _audit(str(OUT), "simulate_override", finding_id=body.finding_id, + policy=body.policy_name, lb=body.lb, block_rate=over.get("block_rate"), + threshold=over.get("threshold"), reason=over.get("reason")) art = str(OUT / "policies" / f"service_policy.{body.policy_name}.json") if body.refine and not body.dry_run: from ..refiner import refine_apply_service_policy diff --git a/src/vpcopilot/console/static/index.html b/src/vpcopilot/console/static/index.html index e2ec60b..b420a53 100644 --- a/src/vpcopilot/console/static/index.html +++ b/src/vpcopilot/console/static/index.html @@ -83,7 +83,7 @@ .stage h4 { margin:0 0 4px; font-size:13px; } .stage .m { font-family:ui-monospace,Menlo,monospace; font-size:11px; color:var(--f5); word-break:break-all; } .stage .r { font-size:12px; color:var(--grey); margin-top:4px; } .arrow { display:flex; align-items:center; color:var(--grey); font-weight:700; padding:0 8px; } .empty { color:var(--grey); padding:8px 0; } - /* ⑥ benchmark dashboard */ + /* ⑦ benchmark dashboard */ .bcards { display:grid; grid-template-columns:repeat(auto-fit,minmax(170px,1fr)); gap:12px; margin-bottom:16px; } .bcard { background:#fff; border:1px solid var(--line); border-radius:12px; padding:14px 15px; } .bcard.win { border-color:#bcd8ec; background:#eef5fb; } @@ -164,7 +164,7 @@

② Review findings

-

What the copilot found and verified, and the XC control it recommends for each. Click a row to inspect the exploit, code, and generated policy. Ready? Go to ③ Mitigate.

+

What the copilot found and verified, and the XC control it recommends for each. Click a row to inspect the exploit, code, and generated policy. Ready? Go to ④ Mitigate.

@@ -176,26 +176,46 @@
- + +
+

③ Simulate the blast radius

+

The safety spine proves a band-aid blocks the finding's exploit and passes one legit request. It says nothing about the other million requests a day — and that gap is why controls sit in monitor mode. This replays a recorded traffic sample against each candidate through a spare load balancer (set it in Run settings) and reports what each policy would block, before anything touches production. Nothing is applied here; the spare LB is snapshotted and restored.

+
+
+
+
+
+
+
+ +
+ + +
+
Loading…
+
+
+ +
-

③ Mitigate live with XC

+

④ Mitigate live with XC

Apply each recommended band-aid to the load balancer and validate it against the finding's real exploit — attach → validate → refine → keep or roll back. Turn off dry-run in Run settings to make it live. “✓ covered by …” means a sibling finding's band-aid already protects this one (same file + control). Only per-request controls (service_policy, api_schema) block a single fired exploit; rate-limit / bot / malicious-user / data-guard are behavioral and validate at config level.

Loading…
- +
-

④ Open the code fix (the cure)

+

⑤ Open the code fix (the cure)

Band-aids are temporary. Every finding also gets a real code-fix PR. Set a PR repo in Run settings first.

Loading…
- +
-

⑤ Retire band-aids

+

⑥ Retire band-aids

Once a cure ships, detach its temporary XC control. The ledger tracks every finding found → mitigated → remediated → retired.

Loading…
@@ -211,10 +231,10 @@
- +
-

⑥ Benchmark — compare models

-

Build a model-tagged report from the current run (discovery + generated policies + live policy quality from ③ Mitigate), then compare Claude / OpenAI / your local model side by side. Workflow: pick a model ▸ scan into out-<model>-<app> ▸ Mitigate ALL ▸ Build benchmark here — repeat per model.

+

⑦ Benchmark — compare models

+

Build a model-tagged report from the current run (discovery + generated policies + live policy quality from ④ Mitigate), then compare Claude / OpenAI / your local model side by side. Workflow: pick a model ▸ scan into out-<model>-<app> ▸ Mitigate ALL ▸ Build benchmark here — repeat per model.

@@ -247,8 +267,8 @@
diff --git a/src/vpcopilot/export.py b/src/vpcopilot/export.py index f15c6eb..9841e5f 100644 --- a/src/vpcopilot/export.py +++ b/src/vpcopilot/export.py @@ -49,7 +49,8 @@ # no findings.json, a run with no live apply has no lb_snapshot.json. BUNDLE_FILES = ["run.json", "audit.log", "ledger.json", "findings.json", "triage.json", "policies.json", "remediations.json", "summary.json", "metrics.json", - "probes.json", "correlations.json", "lb_snapshot.json", "report.html"] + "probes.json", "correlations.json", "lb_snapshot.json", "simulation.json", + "report.html"] def _rj(out: Path, name: str, default): @@ -174,6 +175,9 @@ def build_manifest(out_dir: str = "out", *, members: dict | None = None) -> dict "apply; a CLI-driven run has none.", "Entries written by older builds may lack finding_id / namespace / actor — those cells are " "blank rather than inferred.", + "simulation.json, when present, describes ONE recorded sample replayed through a spare " + "load balancer — not production traffic in general. Its records are redacted at ingest " + "(see `redacted`); read the window and record count with the rate.", ], "members": members or {}, } diff --git a/src/vpcopilot/report.py b/src/vpcopilot/report.py index 643b062..0f94078 100644 --- a/src/vpcopilot/report.py +++ b/src/vpcopilot/report.py @@ -258,6 +258,39 @@ def _models_html() -> str: f'
{chips}
') +def _blast_radius_html(out_dir: str) -> str: + """G2: what each band-aid would do to recorded traffic, beside the exploit before/after. The + impact table answers "does it block the attack"; this answers "what else does it block".""" + try: + from .simulate import load_result + sim = load_result(out_dir) + except Exception: # noqa: BLE001 + return "" + pols = (sim or {}).get("policies") or [] + if not pols: + return "" + rows = "" + for p in pols: + rate = f'{(p.get("block_rate") or 0) * 100:.1f}%' + verdict = ('over threshold' if p.get("blocked_promotion") + else ('error' if p.get("error") + else 'within threshold')) + top = ", ".join(f'{_e(t[0])} ×{_e(t[1])}' for t in (p.get("top_paths") or [])[:3]) or "—" + rows += (f'{_e(p.get("policy_name"))}' + f'{_e(p.get("evaluated"))}{_e(p.get("would_block"))}' + f'{rate}{verdict}{top}') + meta = (f'{_e(sim.get("records_replayed"))} of {_e(sim.get("records"))} recorded request(s) ' + f'replayed through {_e(sim.get("lb"))}') + if sim.get("window"): + meta += f' · window {_e(sim["window"])}' + if sim.get("bodies_available") is False: + meta += ' · no request bodies in this sample' + return ('

Blast radius what each band-aid would block in recorded ' + f'traffic — {meta}

' + '' + f'{rows}
policyevaluatedwould blockrateverdicttop blocked paths
') + + def build_report(out_dir: str = "out") -> str: out = Path(out_dir) summary = _load(out, "summary.json", {}) @@ -322,6 +355,7 @@ def build_report(out_dir: str = "out") -> str: '' '' f'{impact}
controlpolicyexploit beforeexploit afterlegitresultwhen
') if impact else "" + impact_html += _blast_radius_html(out_dir) ts = datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC") try: diff --git a/src/vpcopilot/schemas.py b/src/vpcopilot/schemas.py index 25b313a..063c8c4 100644 --- a/src/vpcopilot/schemas.py +++ b/src/vpcopilot/schemas.py @@ -163,3 +163,60 @@ class RefinedPolicy(BaseModel): False, description="true only if this control genuinely cannot block the exploit without over-blocking" ) recommend: str = Field("", description="if unfixable: the control to use instead, or 'code_fix_only'") + + +# ---- G2: shadow simulation -------------------------------------------------------------- +class RequestRecord(BaseModel): + """One observed request, replayed against a candidate band-aid to measure blast radius. + + Deliberately `ProbeRequest`-shaped so a finding's own exploit/legit requests are valid records + for free. `json_body` is None for records from XC access logs — XC does not capture request + bodies — which is why a `body_matcher` policy cannot be judged from tenant logs alone.""" + method: str = "GET" + path: str = Field(..., description="path only; any query string is split into `query`") + query: dict[str, list[str]] = Field(default_factory=dict) + headers: dict[str, str] = Field(default_factory=dict) + json_body: dict | None = None + ts: str | None = None + status: int | None = Field(None, description="the status the origin gave when observed") + user_agent: str = "" + source: str = Field("", description="which ingest produced it: har | jsonl | xc") + + +class PolicySimulation(BaseModel): + """What one candidate policy would do to a recorded traffic sample.""" + policy_name: str + control: str = "service_policy" + finding_id: str = "" + evaluated: int = 0 + would_block: int = 0 + block_rate: float = 0.0 + threshold: float = 0.0 + blocked_promotion: bool = Field( + False, description="block_rate exceeded the threshold — surfaced at the gate, overridable") + enforcement_confirmed: bool = Field( + False, description="the finding's exploit was observed blocked before counting, so the " + "sample was measured against a policy the edge is actually enforcing") + reason: str = "" + errored: int = Field(0, description="replayed requests that failed in transit — excluded from " + "`evaluated`, since a rate is only honest over requests " + "that reached the edge") + samples: list[dict] = Field(default_factory=list, description="a capped sample of blocked requests") + top_paths: list[list] = Field(default_factory=list, description="[[path, count], ...] of blocks") + top_user_agents: list[list] = Field(default_factory=list) + error: str = "" + + +class SimulationResult(BaseModel): + """The `/simulation.json` artifact.""" + ts: str = "" + lb: str = Field("", description="the LB the replay ran through") + source: str = "" + records: int = 0 + records_replayed: int = 0 + window: str = "" + redacted: dict[str, int] = Field(default_factory=dict, description="field -> count of values stripped") + bodies_available: bool = Field( + True, description="False when records came from XC logs, which capture no request body") + policies: list[PolicySimulation] = Field(default_factory=list) + caveats: list[str] = Field(default_factory=list) diff --git a/src/vpcopilot/simulate.py b/src/vpcopilot/simulate.py new file mode 100644 index 0000000..6cc1974 --- /dev/null +++ b/src/vpcopilot/simulate.py @@ -0,0 +1,281 @@ +"""G2 — shadow simulation: what would this band-aid do to everything *else*? + +The safety spine proves a policy blocks the finding's exploit and passes one legit request. It says +nothing about the other million requests a day, and that gap is why security teams leave controls in +monitor mode. This replays a recorded sample against each candidate **through a spare LB** and +reports the would-block set before anything reaches the human gate. + +Replay runs on a real LB rather than an offline evaluator on purpose (roadmap G1, decided): XC's own +enforcement is the only oracle that cannot silently disagree with XC. `apply.py` does not send what +`generate` emits — `normalize_service_policy_spec` fills defaults that decide the verdict — so a +second implementation would have to track that forever, and a wrong "allow" would ship the outage +this feature exists to prevent. + +Safety: the spare LB is snapshotted and restored in a `finally`, the same spine `apply` uses; a +replay failure is reported on the policy, never raised past the restore. Nothing here writes an +audit record — a rehearsal is not a change to answer for.""" +from __future__ import annotations + +import json +from collections import Counter +from pathlib import Path +from typing import Callable + +import httpx + +from .engine import ApplyContext, guard_lb, safe_rollback +from .probe import _blocked +from .runmeta import utc_now +from .schemas import PolicySimulation, RequestRecord, SimulationResult +from .traffic import from_probe_request +from .xc import XC + +MAX_SAMPLES = 10 # blocked requests carried into the report, per policy +DEFAULT_THRESHOLD = 0.01 # 1% of the sample — override with --threshold / VPCOPILOT_SIM_THRESHOLD +_SP_ONEOF = ("no_service_policies", "active_service_policies", "service_policies_from_namespace") + + +def _fire(rec: RequestRecord, url: str) -> tuple[int, str]: + """Replay one record. Module-level so tests can substitute a transport with no network.""" + r = httpx.request(rec.method, url.rstrip("/") + rec.path, params=rec.query or None, + headers=rec.headers or None, json=rec.json_body, timeout=15, + follow_redirects=False) + return r.status_code, r.text + + +def _try_fire(rec: RequestRecord, url: str) -> tuple[int | None, str]: + """Per-request error isolation. A replay is hundreds of live requests over a network; one + reset connection or TLS hiccup must not discard the whole measurement. An errored request is + counted separately and is NEITHER blocked nor allowed — guessing either way would bias the + number this feature exists to report honestly. (Caught live: one transient SSL EOF aborted an + entire policy's simulation.)""" + try: + return _fire(rec, url) + except Exception: # noqa: BLE001 — every transport failure is per-request, never fatal + return None, "" + + +def sim_name(policy_name: str) -> str: + """Simulation always runs against a throwaway object, never the operator's. + + Attaching by name alone is not enough: a candidate straight from a scan has no object in the + tenant yet, and referencing a policy that does not exist breaks the LB's config outright + (caught live — every replayed request failed in transit, not just the ones under the rule). + Creating under our own name also guarantees we measure the artifact on disk rather than + whatever an identically-named object happens to contain, and makes cleanup unambiguous: + we created it, so we may delete it.""" + return f"{policy_name}-vpcsim" + + +def _create_sim_policy(xc, name: str, spec: dict, log: Callable) -> str: + from .apply import PROTECTED_POLICIES, artifact_spec, normalize_service_policy_spec + tmp = sim_name(name) + if tmp in PROTECTED_POLICIES or name in PROTECTED_POLICIES: + raise RuntimeError(f"refusing to simulate against protected policy '{name}'") + body = {"metadata": {"name": tmp, "namespace": xc.ns}, + "spec": normalize_service_policy_spec(artifact_spec(spec))} + if xc.service_policy_exists(tmp): + xc.put_service_policy(tmp, body) # a leftover from an interrupted run — replace it + else: + xc.create_service_policy(body) + log(f" created throwaway policy '{tmp}'") + return tmp + + +def _attach(ctx: ApplyContext, xc, policy_name: str) -> None: + spec = {k: v for k, v in ctx.spec.items() if k not in _SP_ONEOF} + spec["active_service_policies"] = {"policies": [{"namespace": xc.ns, "name": policy_name}]} + ctx.put(spec) + + +def _canary(out_dir: str, finding_id: str) -> RequestRecord | None: + """The finding's own exploit, as a record — a request we KNOW the policy must block. Used to + confirm enforcement is live before counting anything.""" + p = Path(out_dir) / "probes.json" + if not finding_id or not p.exists(): + return None + try: + for pr in json.loads(p.read_text()): + if pr.get("finding_id") == finding_id and pr.get("exploit"): + return from_probe_request(pr["exploit"], source="probe") + except (json.JSONDecodeError, TypeError): + return None + return None + + +def _await_enforcement(url: str, canary: RequestRecord | None, log: Callable, *, + wait_seconds: int, retries: int, sleep) -> bool: + """XC config takes seconds to reach the edge. Replaying before then counts an unenforced + policy as harmless — the failure mode that makes a blast-radius number a lie. + + With a canary (the finding's own exploit) we poll until it is actually blocked, so the count + that follows is measured against a live policy. Without one, wait the same fixed interval the + apply path uses and say so, because we cannot prove enforcement.""" + if canary is None: + sleep(wait_seconds) + log(f" waited {wait_seconds}s for propagation (no probe for this finding — enforcement unconfirmed)") + return False + for i in range(1, retries + 1): + sleep(wait_seconds) + status, text = _try_fire(canary, url) + if status is not None and _blocked(status, text): + log(f" enforcing after ~{i * wait_seconds}s (canary {canary.method} {canary.path} -> {status})") + return True + log(f" ⚠ canary never blocked after {retries * wait_seconds}s — the policy may not be enforcing") + return False + + +def _replay(records: list[RequestRecord], url: str, + log: Callable) -> tuple[int, list[RequestRecord], int]: + """Returns (evaluated, blocked, errored). `evaluated` excludes errored requests: a rate is only + honest over requests that actually reached the edge.""" + blocked, errored = [], 0 + for rec in records: + status, text = _try_fire(rec, url) + if status is None: + errored += 1 + continue + if _blocked(status, text): + blocked.append(rec) + return len(records) - errored, blocked, errored + + +def simulate_policies(candidates: list[dict], records: list[RequestRecord], *, lb: str, url: str, + out_dir: str = "out", threshold: float = DEFAULT_THRESHOLD, + max_records: int | None = None, source: str = "", window: str = "", + redacted: dict | None = None, keep_policy: bool = False, + wait_seconds: int = 6, retries: int = 8, + log: Callable = print) -> SimulationResult: + """Replay `records` against each candidate on `lb`, one policy at a time. + + One at a time is deliberate: two candidates on the same LB share FIRST_MATCH ordering, so a + batched run would measure something no single band-aid does.""" + xc = XC() + guard_lb(lb, allow_protected=False, dry_run=False) + sample = records[:max_records] if max_records else records + bodies = not (sample and all(r.source == "xc" for r in sample)) + + res = SimulationResult( + ts=utc_now(), lb=lb, source=source, records=len(records), records_replayed=len(sample), + window=window, redacted=dict(redacted or {}), bodies_available=bodies, + caveats=[ + "A would-block count describes THIS sample only — a quiet window makes any policy look " + "safe. The window and record count are stated above; read them together with the rate.", + "Replay runs one policy at a time on a spare LB. Two candidates share FIRST_MATCH " + "ordering, so a batched result would not describe either band-aid alone.", + ]) + if not bodies: + res.caveats.append( + "Records came from XC access logs, which capture no request body — a `body_matcher` " + "policy cannot be judged from them. Supply a HAR to exercise body predicates.") + + for cand in candidates: + name = cand["policy_name"] + sim = PolicySimulation(policy_name=name, control=cand.get("control", "service_policy"), + finding_id=cand.get("finding_id", ""), threshold=threshold) + ctx = ApplyContext(xc=xc, lb=lb, out_dir=out_dir, log=log, + finding_id=cand.get("finding_id")).load() + tmp = None + try: + ctx.self_test() + tmp = _create_sim_policy(xc, name, cand.get("spec") or {}, log) + _attach(ctx, xc, tmp) + log(f"simulate: '{name}' attached to {lb} · waiting for the edge to enforce it") + sim.enforcement_confirmed = _await_enforcement( + url, _canary(out_dir, cand.get("finding_id", "")), log, + wait_seconds=wait_seconds, retries=retries, sleep=ctx.sleep) + log(f" replaying {len(sample)} record(s)") + total, blocked, errored = _replay(sample, url, log) + sim.errored = errored + if errored: + log(f" {errored} record(s) failed in transit — excluded from the rate") + sim.evaluated = total + sim.would_block = len(blocked) + sim.block_rate = round(len(blocked) / total, 4) if total else 0.0 + sim.samples = [b.model_dump(exclude={"headers"}) for b in blocked[:MAX_SAMPLES]] + sim.top_paths = [list(x) for x in Counter(b.path for b in blocked).most_common(5)] + sim.top_user_agents = [list(x) for x in + Counter(b.user_agent for b in blocked if b.user_agent).most_common(5)] + if not total: + sim.reason = ("nothing measured — every replayed request failed in transit, so this " + "is zero evidence, not a clean result") + if total and sim.block_rate > threshold: + sim.blocked_promotion = True + sim.reason = (f"would block {sim.would_block}/{total} recorded requests " + f"({sim.block_rate:.1%}), over the {threshold:.1%} threshold") + log(f" ⚠ {name}: {sim.reason}") + else: + log(f" {name}: would block {sim.would_block}/{total}") + except Exception as e: # noqa: BLE001 — a replay failure is reported, never raised past restore + sim.error = str(e) + log(f" ⚠ {name}: simulation failed — {e}") + finally: + if not keep_policy: + safe_rollback(ctx, verify=lambda b: "active_service_policies" not in b) + if tmp: # we created it, so we clean it up — detach first, then delete + try: + xc.delete_service_policy(tmp) + log(f" deleted throwaway policy '{tmp}'") + except Exception as e: # noqa: BLE001 + log(f" ⚠ could not delete '{tmp}': {e}") + res.policies.append(sim) + if any(p.errored for p in res.policies): + res.caveats.append( + "Some replayed requests failed in transit and are excluded from the rate — the " + "`errored` count says how many. A rate over a shrunken sample is weaker evidence.") + if any(not p.enforcement_confirmed and not p.error for p in res.policies): + res.caveats.append( + "Enforcement could not be CONFIRMED for at least one policy (no probe to use as a " + "canary). A would-block count measured before the edge picks up the config reads as " + "harmless — treat an unconfirmed 0% as unmeasured, not as safe.") + return res + + +def candidates_from_out(out_dir: str, policy: str | None = None) -> list[dict]: + """The scan's generated service policies, from the `policies.json` index. Only `service_policy` + is replayable today: the other six controls are parameterized by the engine rather than matched + per request, so there is no per-request verdict to count.""" + out = Path(out_dir) + idx = out / "policies.json" + if not idx.exists(): + return [] + cands = [] + for a in json.loads(idx.read_text()): + if a.get("control") != "service_policy": + continue + if policy and a.get("policy_name") != policy: + continue + art = out / "policies" / f"service_policy.{a['policy_name']}.json" + if not art.exists(): + continue + cands.append({"policy_name": a["policy_name"], "control": "service_policy", + "finding_id": a.get("finding_id", ""), "spec": json.loads(art.read_text())}) + return cands + + +def write_result(out_dir: str, res: SimulationResult) -> str: + p = Path(out_dir) / "simulation.json" + p.parent.mkdir(parents=True, exist_ok=True) + p.write_text(json.dumps(res.model_dump(), indent=2)) + return str(p) + + +def load_result(out_dir: str) -> dict: + p = Path(out_dir) / "simulation.json" + if not p.exists(): + return {} + try: + return json.loads(p.read_text()) + except json.JSONDecodeError: + return {} + + +def promotion_block(out_dir: str, policy_name: str) -> dict | None: + """The gate check: has this policy been simulated and found too broad? + + Returns None when there is no simulation for it — G2 adds a warning, **not** a prerequisite. + An operator who never runs `simulate` sees exactly the behaviour they see today.""" + for p in load_result(out_dir).get("policies") or []: + if p.get("policy_name") == policy_name and p.get("blocked_promotion"): + return p + return None diff --git a/src/vpcopilot/traffic.py b/src/vpcopilot/traffic.py new file mode 100644 index 0000000..3b49514 --- /dev/null +++ b/src/vpcopilot/traffic.py @@ -0,0 +1,168 @@ +"""G2 — turn recorded traffic into `RequestRecord`s a candidate band-aid can be replayed against. + +Three sources, one shape: a HAR export, a JSONL sample, or XC access logs. A HAR carries request +bodies; **XC access logs do not**, which is why a `body_matcher` policy cannot be judged from +tenant logs alone — `SimulationResult.bodies_available` says so out loud rather than letting a +would-block count quietly mean less than it looks. + +Redaction runs here, at ingest, and not at export. The redacted sample ships inside the evidence +bundle, so a secret that is never parsed into a record can never leak from one; a filter applied +later would only cover the paths someone remembered to route through it. Every ingest returns +`(records, redaction_counts)` and the counts reach `simulation.json`, because a redaction you +cannot see is a redaction you cannot trust. + +Read-only: nothing here contacts XC, and nothing mutates the source it reads.""" +from __future__ import annotations + +import json +from pathlib import Path +from urllib.parse import parse_qs, urlsplit + +from .schemas import RequestRecord + +REDACTED = "[redacted]" + +# Stripped from headers entirely — the name alone is not sensitive, the value always is. +REDACT_HEADERS = ("authorization", "cookie", "set-cookie", "proxy-authorization", + "x-api-key", "x-auth-token", "api-key") +# Replaced (not dropped) in a JSON body: the key must survive so an `additionalProperties:false` +# or a required-field matcher still sees the same document shape. +REDACT_BODY_KEYS = ("password", "passwd", "secret", "token", "access_token", "refresh_token", + "api_key", "apikey", "authorization", "ssn", "card", "card_number", "cvv", "pan") + + +def _split(url_or_path: str) -> tuple[str, dict]: + """`/a/b?x=1&x=2` -> (`/a/b`, {'x': ['1','2']}). Accepts a full URL or a bare path.""" + parts = urlsplit(url_or_path or "") + return (parts.path or "/"), parse_qs(parts.query, keep_blank_values=True) + + +def _clean_headers(headers: dict, extra: tuple, counts: dict) -> dict: + out = {} + drop = tuple(h.lower() for h in REDACT_HEADERS) + tuple(h.lower() for h in extra) + for k, v in (headers or {}).items(): + if str(k).lower() in drop: + counts[str(k).lower()] = counts.get(str(k).lower(), 0) + 1 + continue + out[k] = v + return out + + +def _clean_body(body, counts: dict): + """Replace secret-looking values in place, keeping every key — the document shape is what a + schema or body matcher is judged against.""" + if isinstance(body, dict): + out = {} + for k, v in body.items(): + if str(k).lower() in REDACT_BODY_KEYS and not isinstance(v, (dict, list)): + counts[str(k).lower()] = counts.get(str(k).lower(), 0) + 1 + out[k] = REDACTED + else: + out[k] = _clean_body(v, counts) + return out + if isinstance(body, list): + return [_clean_body(v, counts) for v in body] + return body + + +def _record(*, method, url_or_path, headers, body, ts, status, source, + redact_headers=(), counts=None) -> RequestRecord: + counts = counts if counts is not None else {} + path, query = _split(url_or_path) + hdrs = _clean_headers(headers, redact_headers, counts) + ua = next((v for k, v in hdrs.items() if str(k).lower() == "user-agent"), "") + return RequestRecord(method=(method or "GET").upper(), path=path, query=query, headers=hdrs, + json_body=_clean_body(body, counts) if isinstance(body, (dict, list)) else None, + ts=ts, status=status, user_agent=ua, source=source) + + +# ---------------- sources ---------------- +def from_har(har: dict, *, redact_headers=()) -> tuple[list[RequestRecord], dict]: + counts: dict = {} + out = [] + for e in ((har or {}).get("log") or {}).get("entries") or []: + req = e.get("request") or {} + hdrs = {h.get("name"): h.get("value") for h in (req.get("headers") or []) if h.get("name")} + body = None + post = req.get("postData") or {} + if "json" in (post.get("mimeType") or "") and post.get("text"): + try: + body = json.loads(post["text"]) + except json.JSONDecodeError: + body = None + out.append(_record(method=req.get("method"), url_or_path=req.get("url"), headers=hdrs, + body=body, ts=e.get("startedDateTime"), + status=(e.get("response") or {}).get("status"), source="har", + redact_headers=redact_headers, counts=counts)) + return out, counts + + +def from_xc_logs(logs: list, *, lb: str | None = None, redact_headers=()) -> tuple[list[RequestRecord], dict]: + """Map XC access-log records. `vh_name` is `ves-io-http-loadbalancer-`, so a run can be + scoped to the LB it cares about. `src_ip` is deliberately NOT carried into the record: the + sample is distributed in the evidence bundle and nothing in G2 needs a client address.""" + counts: dict = {} + out = [] + for raw in logs or []: + rec = raw + if isinstance(rec, str): + try: + rec = json.loads(rec) + except json.JSONDecodeError: + counts["_skipped"] = counts.get("_skipped", 0) + 1 + continue + if lb and not str(rec.get("vh_name", "")).endswith(f"-{lb}"): + continue + hdrs = rec.get("req_headers") or {} + if isinstance(hdrs, str): + try: + hdrs = json.loads(hdrs) + except json.JSONDecodeError: + hdrs = {} + if rec.get("user_agent") and not any(k.lower() == "user-agent" for k in hdrs): + hdrs["User-Agent"] = rec["user_agent"] + out.append(_record(method=rec.get("method"), url_or_path=rec.get("req_path") or rec.get("original_path"), + headers=hdrs, body=None, ts=rec.get("time") or rec.get("@timestamp"), + status=rec.get("rsp_code"), source="xc", + redact_headers=redact_headers, counts=counts)) + return out, counts + + +def from_jsonl(text: str, *, redact_headers=()) -> tuple[list[RequestRecord], dict]: + """One JSON object per line, already in `RequestRecord` shape (what `--logs` usually gets). + A malformed line is counted and skipped — one bad line must never kill a replay.""" + counts: dict = {} + out = [] + for ln in (text or "").splitlines(): + if not ln.strip(): + continue + try: + d = json.loads(ln) + out.append(_record(method=d.get("method"), url_or_path=d.get("path") or d["path"], + headers=d.get("headers") or {}, body=d.get("json_body"), + ts=d.get("ts"), status=d.get("status"), source=d.get("source") or "jsonl", + redact_headers=redact_headers, counts=counts)) + except (json.JSONDecodeError, KeyError, TypeError): + counts["_skipped"] = counts.get("_skipped", 0) + 1 + return out, counts + + +def from_probe_request(pr: dict, *, source: str = "probe") -> RequestRecord: + """A finding's own exploit/legit request is a valid record — `ProbeRequest` is the shape + `RequestRecord` was modelled on, so the acceptance case needs no second fixture.""" + counts: dict = {} + return _record(method=pr.get("method"), url_or_path=pr.get("path"), headers=pr.get("headers") or {}, + body=pr.get("json_body"), ts=None, status=None, source=source, counts=counts) + + +def load(path: str, *, redact_headers=()) -> tuple[list[RequestRecord], dict]: + """Dispatch on extension: `.har` / `.json` (HAR) or `.jsonl` / `.ndjson`.""" + p = Path(path) + suffix = p.suffix.lower() + if suffix not in (".har", ".json", ".jsonl", ".ndjson"): # checked BEFORE reading, so an + raise ValueError( # unsupported type says so plainly + f"unsupported traffic file '{p.suffix}' — use .har, .json, .jsonl or .ndjson") + text = p.read_text() + if suffix in (".har", ".json"): + return from_har(json.loads(text), redact_headers=redact_headers) + return from_jsonl(text, redact_headers=redact_headers) diff --git a/src/vpcopilot/xc.py b/src/vpcopilot/xc.py index b965b11..f69d154 100644 --- a/src/vpcopilot/xc.py +++ b/src/vpcopilot/xc.py @@ -91,6 +91,25 @@ def list_http_loadbalancers(self) -> dict: def get_lb(self, name: str) -> dict: return self._req("GET", f"/config/namespaces/{self.ns}/http_loadbalancers/{name}") + # --- request logs (G2: read-only; the ONLY /data endpoint this client touches) --- + def access_logs(self, *, start: str, end: str, limit: int = 500, lb: str | None = None) -> list[dict]: + """Observed requests from the tenant, for shadow simulation. `start`/`end` are RFC3339 + (`2026-07-27T15:00:00Z`) — XC rejects relative times like `now-1h`. Optionally scoped to + one LB via its virtual-host name. Records carry no request body; see `traffic.py`.""" + body = {"start_time": start, "end_time": end, "limit": int(limit), "scroll": False} + if lb: + body["query"] = f'{{vh_name="ves-io-http-loadbalancer-{lb}"}}' + res = self._req("POST", f"/data/namespaces/{self.ns}/access_logs", json=body) + out = [] + for row in res.get("logs") or []: + if isinstance(row, str): + try: + row = json.loads(row) + except json.JSONDecodeError: + continue # one unparseable row must not lose the rest of the window + out.append(row) + return out + def put_lb(self, name: str, obj: dict) -> dict: return self._req("PUT", f"/config/namespaces/{self.ns}/http_loadbalancers/{name}", json=obj) diff --git a/tests/conftest.py b/tests/conftest.py index 96846ee..35b48d6 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -53,6 +53,9 @@ def put_service_policy(self, name, obj): self.service_policies[name] = copy.deepcopy(obj) return obj + def delete_service_policy(self, name): + return self.service_policies.pop(name, None) + # ---- app firewall ---- def app_firewall_exists(self, name): return name in self.app_firewalls diff --git a/tests/test_console_simulate.py b/tests/test_console_simulate.py new file mode 100644 index 0000000..34ff40a --- /dev/null +++ b/tests/test_console_simulate.py @@ -0,0 +1,114 @@ +"""G2 console surface: the Simulate step reads /api/simulate, and a policy simulation flagged as +too broad WARNS at the gate rather than silently applying — or silently blocking.""" +import json + +from fastapi.testclient import TestClient + +from vpcopilot import audit +from vpcopilot.console import app as A + + +def _client(): + return TestClient(A.app) + + +def _sim(out, **over): + (out / "simulation.json").write_text(json.dumps({ + "ts": "2026-07-27T12:00:00Z", "lb": "vpcopilot-lab", "records": 200, "records_replayed": 200, + "bodies_available": True, "caveats": ["sample-scoped"], + "policies": [{"policy_name": "deny-wide", "blocked_promotion": True, "block_rate": 0.9, + "threshold": 0.01, "reason": "would block 180/200 recorded requests (90.0%), " + "over the 1.0% threshold", **over}, + {"policy_name": "deny-narrow", "blocked_promotion": False, "block_rate": 0.0, + "threshold": 0.01, "reason": ""}]})) + + +# ---- read ---- +def test_simulate_endpoint_returns_the_last_result(tmp_path, monkeypatch): + _sim(tmp_path) + monkeypatch.setattr(A, "OUT", tmp_path) + body = _client().get("/api/simulate").json() + assert body["out"] == str(tmp_path) + assert body["simulation"]["policies"][0]["policy_name"] == "deny-wide" + + +def test_simulate_endpoint_is_empty_before_any_run(tmp_path, monkeypatch): + monkeypatch.setattr(A, "OUT", tmp_path / "never") + r = _client().get("/api/simulate") + assert r.status_code == 200 and r.json()["simulation"] == {} + + +# ---- the gate: warn + explicit override, audited ---- +def test_an_overbroad_policy_is_refused_without_the_override(tmp_path, monkeypatch): + _sim(tmp_path) + monkeypatch.setattr(A, "OUT", tmp_path) + r = _client().post("/api/action", json={"control": "service_policy", "policy_name": "deny-wide", + "finding_id": "f-1", "lb": "lab", "dry_run": False}) + assert r.status_code == 200 # the job starts… + job = r.json()["job"] + for _ in range(50): + s = _client().get(f"/api/action?job={job}").json() + if s["state"] != "running": + break + assert s["state"] == "error" and "allow overbroad" in s["error"] + + +def test_the_override_applies_anyway_and_writes_an_audit_record(tmp_path, monkeypatch): + _sim(tmp_path) + monkeypatch.setattr(A, "OUT", tmp_path) + monkeypatch.setattr(A, "_dispatch_action", lambda body, log: {"passed": True, "kept": True}) + r = _client().post("/api/action", json={"control": "service_policy", "policy_name": "deny-wide", + "finding_id": "f-1", "lb": "lab", "dry_run": False, + "allow_overbroad": True}) + job = r.json()["job"] + for _ in range(50): + s = _client().get(f"/api/action?job={job}").json() + if s["state"] != "running": + break + assert s["state"] == "done" + + +def test_a_narrow_policy_is_not_gated(tmp_path, monkeypatch): + _sim(tmp_path) + monkeypatch.setattr(A, "OUT", tmp_path) + monkeypatch.setattr(A, "_dispatch_action", lambda body, log: {"passed": True}) + r = _client().post("/api/action", json={"control": "service_policy", "policy_name": "deny-narrow", + "finding_id": "f-2", "lb": "lab", "dry_run": False}) + job = r.json()["job"] + for _ in range(50): + s = _client().get(f"/api/action?job={job}").json() + if s["state"] != "running": + break + assert s["state"] == "done" + + +def test_a_dry_run_is_never_gated(tmp_path, monkeypatch): + """Dry-run changes nothing, so a blast-radius warning has nothing to gate.""" + _sim(tmp_path) + monkeypatch.setattr(A, "OUT", tmp_path) + monkeypatch.setattr(A, "_dispatch_action", lambda body, log: {"mode": "dry_run"}) + r = _client().post("/api/action", json={"control": "service_policy", "policy_name": "deny-wide", + "finding_id": "f-1", "lb": "lab", "dry_run": True}) + job = r.json()["job"] + for _ in range(50): + s = _client().get(f"/api/action?job={job}").json() + if s["state"] != "running": + break + assert s["state"] == "done" + + +# ---- no-regression pin ---- +def test_apply_is_unchanged_when_nothing_was_simulated(tmp_path, monkeypatch): + """G2 adds a check, not a prerequisite: with no simulation.json the apply path behaves exactly + as it did before this feature existed.""" + monkeypatch.setattr(A, "OUT", tmp_path) + monkeypatch.setattr(A, "_dispatch_action", lambda body, log: {"passed": True, "kept": True}) + r = _client().post("/api/action", json={"control": "service_policy", "policy_name": "anything", + "finding_id": "f-9", "lb": "lab", "dry_run": False}) + job = r.json()["job"] + for _ in range(50): + s = _client().get(f"/api/action?job={job}").json() + if s["state"] != "running": + break + assert s["state"] == "done" + assert not [e for e in audit.load(str(tmp_path)) if e["action"] == "simulate_override"] diff --git a/tests/test_simulate.py b/tests/test_simulate.py new file mode 100644 index 0000000..9bf84da --- /dev/null +++ b/tests/test_simulate.py @@ -0,0 +1,258 @@ +"""G2 — shadow simulation. Replays a recorded sample against a candidate band-aid on a spare LB +and reports what it WOULD block, before anything reaches the human gate. + +Offline: the LB is FakeXC and the replay is a stub transport, so no tenant and no network.""" +import json + +import pytest + +from vpcopilot import simulate, traffic +from vpcopilot.schemas import RequestRecord + +POLICY = {"metadata": {"name": "deny-negative-pay"}, "spec": {"rule_list": {"rules": [ + {"spec": {"action": "DENY", "path": {"exact_values": ["/api/pay"]}, + "http_method": {"methods": ["POST"]}, + "body_matcher": {"regex_values": ["amount[^0-9-]*-[0-9]"]}}}, + {"spec": {"action": "ALLOW", "path": {"prefix_values": ["/"]}}}, +]}}} + + +def _recs(*paths): + return [RequestRecord(path=p, method="POST") for p in paths] + + +def _fire(blocked_paths): + """Stub replay: XC 403s exactly the paths named, everything else passes through.""" + def fire(rec, url): + return (403, "") if rec.path in blocked_paths else (200, "ok") + return fire + + +@pytest.fixture(autouse=True) +def _fast(monkeypatch, noop_sleep): + import vpcopilot.engine as engine + real = engine.ApplyContext.__post_init__ + + def patched(self): + real(self) + self.sleep = noop_sleep + monkeypatch.setattr(engine.ApplyContext, "__post_init__", patched) + monkeypatch.setenv("VPCOPILOT_PROTECTED_LBS", "nimbus-www") + + +def _sim(monkeypatch, fake_xc, tmp_path, recs, blocked, **kw): + monkeypatch.setattr(simulate, "XC", lambda *a, **k: fake_xc) + monkeypatch.setattr(simulate, "_fire", _fire(blocked)) + return simulate.simulate_policies( + [{"policy_name": "deny-negative-pay", "control": "service_policy", + "finding_id": "f-1", "spec": POLICY}], + recs, lb="lab", url="http://lab.test", out_dir=str(tmp_path), log=lambda m: None, **kw) + + +# ---- the acceptance case ---- +def test_surgical_policy_blocks_the_exploit_and_no_legitimate_traffic(monkeypatch, fake_xc, tmp_path): + recs = _recs("/api/pay", "/api/accounts", "/health", "/api/pay") + res = _sim(monkeypatch, fake_xc, tmp_path, recs, blocked={"/api/pay"}) + p = res.policies[0] + assert p.evaluated == 4 and p.would_block == 2 + assert p.block_rate == 0.5 and p.blocked_promotion is True # 50% >> default threshold + + +def test_a_policy_blocking_only_the_exploit_path_stays_under_threshold(monkeypatch, fake_xc, tmp_path): + recs = _recs("/api/pay", *[f"/legit/{i}" for i in range(199)]) + res = _sim(monkeypatch, fake_xc, tmp_path, recs, blocked={"/api/pay"}) + p = res.policies[0] + assert p.would_block == 1 and p.blocked_promotion is False and p.reason == "" + + +def test_an_overbroad_policy_trips_the_threshold_with_a_reason(monkeypatch, fake_xc, tmp_path): + recs = _recs("/a", "/b", "/c", "/d") + res = _sim(monkeypatch, fake_xc, tmp_path, recs, blocked={"/a", "/b", "/c", "/d"}) + p = res.policies[0] + assert p.block_rate == 1.0 and p.blocked_promotion is True + assert "1.0" in p.reason or "100" in p.reason + + +def test_threshold_is_configurable(monkeypatch, fake_xc, tmp_path): + recs = _recs("/api/pay", "/x", "/y", "/z") + assert _sim(monkeypatch, fake_xc, tmp_path, recs, blocked={"/api/pay"}, + threshold=0.5).policies[0].blocked_promotion is False + + +# ---- what the reviewer sees ---- +def test_blocked_samples_and_top_paths_are_reported(monkeypatch, fake_xc, tmp_path): + recs = _recs("/api/pay", "/api/pay", "/admin") + p = _sim(monkeypatch, fake_xc, tmp_path, recs, blocked={"/api/pay", "/admin"}).policies[0] + assert p.top_paths[0] == ["/api/pay", 2] + assert p.samples and p.samples[0]["path"] in ("/api/pay", "/admin") + + +def test_sample_of_blocked_requests_is_capped(monkeypatch, fake_xc, tmp_path): + recs = _recs(*[f"/p{i}" for i in range(100)]) + p = _sim(monkeypatch, fake_xc, tmp_path, recs, blocked={f"/p{i}" for i in range(100)}).policies[0] + assert p.would_block == 100 and len(p.samples) <= simulate.MAX_SAMPLES + + +# ---- safety: the LB must be left exactly as found ---- +def test_the_lb_is_restored_after_a_replay(monkeypatch, fake_xc, tmp_path): + before = json.dumps(fake_xc.lb["spec"], sort_keys=True) + _sim(monkeypatch, fake_xc, tmp_path, _recs("/a"), blocked=set()) + assert json.dumps(fake_xc.lb["spec"], sort_keys=True) == before + + +def test_a_total_network_outage_leaves_the_lb_restored_and_measures_nothing(monkeypatch, fake_xc, tmp_path): + """Every request failing is not a crash and not a 0% block rate — it is zero evidence.""" + before = json.dumps(fake_xc.lb["spec"], sort_keys=True) + monkeypatch.setattr(simulate, "XC", lambda *a, **k: fake_xc) + + def boom(rec, url): + raise RuntimeError("network died mid-replay") + monkeypatch.setattr(simulate, "_fire", boom) + res = simulate.simulate_policies( + [{"policy_name": "p", "control": "service_policy", "finding_id": "f", "spec": POLICY}], + _recs("/a", "/b"), lb="lab", url="http://lab.test", out_dir=str(tmp_path), log=lambda m: None) + p = res.policies[0] + assert p.errored == 2 and p.evaluated == 0 and p.block_rate == 0.0 + assert p.blocked_promotion is False and p.enforcement_confirmed is False + assert json.dumps(fake_xc.lb["spec"], sort_keys=True) == before + + +def test_the_lb_is_restored_when_the_attach_itself_fails(monkeypatch, fake_xc, tmp_path): + """A policy-level failure (not a per-request one) is still reported, never raised past the + restore — the spare LB must never be left carrying a rehearsal.""" + before = json.dumps(fake_xc.lb["spec"], sort_keys=True) + monkeypatch.setattr(simulate, "XC", lambda *a, **k: fake_xc) + monkeypatch.setattr(simulate, "_attach", lambda *a, **k: (_ for _ in ()).throw(RuntimeError("XC 503"))) + res = simulate.simulate_policies( + [{"policy_name": "p", "control": "service_policy", "finding_id": "f", "spec": POLICY}], + _recs("/a"), lb="lab", url="http://lab.test", out_dir=str(tmp_path), log=lambda m: None) + assert "XC 503" in res.policies[0].error + assert json.dumps(fake_xc.lb["spec"], sort_keys=True) == before + + +def test_a_protected_lb_is_refused(monkeypatch, fake_xc, tmp_path): + monkeypatch.setattr(simulate, "XC", lambda *a, **k: fake_xc) + with pytest.raises(RuntimeError, match="protected"): + simulate.simulate_policies([], [], lb="nimbus-www", url="http://x", + out_dir=str(tmp_path), log=lambda m: None) + + +def test_simulation_creates_no_audit_record(monkeypatch, fake_xc, tmp_path): + """Evidence gathering never mutates what it records — a rehearsal is not a change.""" + from vpcopilot import audit + _sim(monkeypatch, fake_xc, tmp_path, _recs("/a"), blocked=set()) + assert audit.load(str(tmp_path)) == [] + + +# ---- artifact + the gate ---- +def test_write_result_produces_the_artifact(monkeypatch, fake_xc, tmp_path): + res = _sim(monkeypatch, fake_xc, tmp_path, _recs("/api/pay"), blocked={"/api/pay"}) + path = simulate.write_result(str(tmp_path), res) + d = json.loads(open(path).read()) + assert d["policies"][0]["policy_name"] == "deny-negative-pay" and d["caveats"] + + +def test_bodies_unavailable_is_recorded_for_xc_sourced_records(monkeypatch, fake_xc, tmp_path): + recs, _ = traffic.from_xc_logs([{"method": "GET", "req_path": "/a", "rsp_code": 200}]) + res = _sim(monkeypatch, fake_xc, tmp_path, recs, blocked=set()) + assert res.bodies_available is False + assert any("body" in c.lower() for c in res.caveats) + + +def test_promotion_block_reads_the_artifact(tmp_path): + (tmp_path / "simulation.json").write_text(json.dumps({"policies": [ + {"policy_name": "wide", "blocked_promotion": True, "block_rate": 0.9, "reason": "too wide"}, + {"policy_name": "ok", "blocked_promotion": False, "block_rate": 0.0}]})) + assert simulate.promotion_block(str(tmp_path), "wide") + assert simulate.promotion_block(str(tmp_path), "ok") is None + + +def test_promotion_block_is_silent_when_nothing_was_simulated(tmp_path): + """No simulation must never block an apply — G2 adds a check, it does not add a prerequisite.""" + assert simulate.promotion_block(str(tmp_path), "anything") is None + + +# ---- propagation: a count taken before the edge enforces is not a measurement ---- +def test_enforcement_is_confirmed_with_the_findings_own_exploit(monkeypatch, fake_xc, tmp_path): + """Caught live: attaching and replaying immediately counted an unenforced policy as harmless — + 0/200 blocked, including the exploit. The canary polls until the policy actually bites.""" + (tmp_path / "probes.json").write_text(json.dumps([ + {"finding_id": "f-1", "exploit": {"method": "POST", "path": "/api/pay", + "json_body": {"amount": -1}}}])) + res = _sim(monkeypatch, fake_xc, tmp_path, _recs("/api/pay", "/health"), blocked={"/api/pay"}) + assert res.policies[0].enforcement_confirmed is True + + +def test_an_unconfirmed_run_says_so_rather_than_reading_as_safe(monkeypatch, fake_xc, tmp_path): + """No probe -> no canary -> a 0% rate is unmeasured, not a clean bill of health.""" + res = _sim(monkeypatch, fake_xc, tmp_path, _recs("/a"), blocked=set()) + assert res.policies[0].enforcement_confirmed is False + assert any("unconfirmed" in c.lower() or "unmeasured" in c.lower() for c in res.caveats) + + +def test_a_canary_that_never_blocks_does_not_confirm(monkeypatch, fake_xc, tmp_path): + (tmp_path / "probes.json").write_text(json.dumps([ + {"finding_id": "f-1", "exploit": {"method": "POST", "path": "/api/pay"}}])) + res = _sim(monkeypatch, fake_xc, tmp_path, _recs("/health"), blocked=set()) + assert res.policies[0].enforcement_confirmed is False + + +def test_one_transport_failure_does_not_discard_the_whole_replay(monkeypatch, fake_xc, tmp_path): + """Caught live: a single transient SSL EOF aborted an entire policy's simulation.""" + calls = {"n": 0} + + def flaky(rec, url): + calls["n"] += 1 + if rec.path == "/boom": + raise RuntimeError("[SSL: UNEXPECTED_EOF_WHILE_READING]") + return (403, "") if rec.path == "/api/pay" else (200, "ok") + monkeypatch.setattr(simulate, "XC", lambda *a, **k: fake_xc) + monkeypatch.setattr(simulate, "_fire", flaky) + res = simulate.simulate_policies( + [{"policy_name": "p", "control": "service_policy", "finding_id": "f", "spec": POLICY}], + _recs("/api/pay", "/boom", "/health"), lb="lab", url="http://lab.test", + out_dir=str(tmp_path), log=lambda m: None) + p = res.policies[0] + assert not p.error # the run completed + assert p.errored == 1 and p.evaluated == 2 and p.would_block == 1 + assert p.block_rate == 0.5 # rate is over what actually reached the edge + assert any("transit" in c.lower() for c in res.caveats) + + +# ---- the object lifecycle: simulate never rides on the operator's policies ---- +def test_a_throwaway_policy_is_created_and_deleted(monkeypatch, fake_xc, tmp_path): + """Caught live: attaching by name alone assumed the object already existed. A candidate + straight from a scan does not, and referencing a missing policy breaks the LB config — + every replayed request failed in transit, not just the ones under the rule.""" + _sim(monkeypatch, fake_xc, tmp_path, _recs("/api/pay"), blocked={"/api/pay"}) + assert simulate.sim_name("deny-negative-pay") == "deny-negative-pay-vpcsim" + assert "deny-negative-pay-vpcsim" not in fake_xc.service_policies # cleaned up + assert "deny-negative-pay" not in fake_xc.service_policies # never created either + + +def test_the_operators_own_policy_is_never_touched(monkeypatch, fake_xc, tmp_path): + fake_xc.service_policies["deny-negative-pay"] = {"metadata": {"name": "deny-negative-pay"}, + "spec": {"_theirs": True}} + _sim(monkeypatch, fake_xc, tmp_path, _recs("/api/pay"), blocked={"/api/pay"}) + assert fake_xc.service_policies["deny-negative-pay"]["spec"] == {"_theirs": True} + + +def test_a_protected_policy_is_refused(monkeypatch, fake_xc, tmp_path): + from vpcopilot.apply import PROTECTED_POLICIES + name = sorted(PROTECTED_POLICIES)[0] + monkeypatch.setattr(simulate, "XC", lambda *a, **k: fake_xc) + monkeypatch.setattr(simulate, "_fire", _fire(set())) + res = simulate.simulate_policies( + [{"policy_name": name, "control": "service_policy", "finding_id": "f", "spec": POLICY}], + _recs("/a"), lb="lab", url="http://lab.test", out_dir=str(tmp_path), log=lambda m: None) + assert "protected" in res.policies[0].error + + +def test_zero_evaluated_is_reported_as_no_evidence_not_as_safe(monkeypatch, fake_xc, tmp_path): + monkeypatch.setattr(simulate, "XC", lambda *a, **k: fake_xc) + monkeypatch.setattr(simulate, "_fire", lambda rec, url: (_ for _ in ()).throw(RuntimeError("down"))) + res = simulate.simulate_policies( + [{"policy_name": "p", "control": "service_policy", "finding_id": "f", "spec": POLICY}], + _recs("/a"), lb="lab", url="http://lab.test", out_dir=str(tmp_path), log=lambda m: None) + p = res.policies[0] + assert p.evaluated == 0 and "zero evidence" in p.reason diff --git a/tests/test_traffic.py b/tests/test_traffic.py new file mode 100644 index 0000000..a593f4a --- /dev/null +++ b/tests/test_traffic.py @@ -0,0 +1,143 @@ +"""G2 — traffic ingestion. Records come from a HAR export, a JSONL sample, or XC access logs, and +are normalized to one `RequestRecord` shape. Redaction runs at INGEST because the redacted sample +ships in the evidence bundle: a value that is never parsed into a record can never leak.""" +import json + +import pytest + +from vpcopilot import traffic + +HAR = {"log": {"entries": [ + {"startedDateTime": "2026-07-27T10:00:00Z", + "request": {"method": "POST", "url": "https://app.example.com/api/pay?ref=abc&ref=def", + "headers": [{"name": "Content-Type", "value": "application/json"}, + {"name": "Authorization", "value": "Bearer sk-live-SECRET"}, + {"name": "User-Agent", "value": "curl/8.1"}], + "postData": {"mimeType": "application/json", "text": '{"amount": -5, "to": "x"}'}}, + "response": {"status": 200}}, + {"request": {"method": "GET", "url": "https://app.example.com/health", "headers": []}, + "response": {"status": 200}}, +]}} + +XC_LOGS = [ + {"method": "GET", "req_path": "/api/accounts?id=7", "rsp_code": 200, "user_agent": "node", + "time": "2026-07-27T15:21:37.178Z", "src_ip": "3.21.170.57", + "req_headers": '{"accept":"*/*","cookie":"session=abc123"}', + "vh_name": "ves-io-http-loadbalancer-nimbus-www"}, + {"method": "POST", "req_path": "/api/pay", "rsp_code": 403, "user_agent": "curl/8", + "time": "2026-07-27T15:22:00.000Z", "req_headers": "{}", + "vh_name": "ves-io-http-loadbalancer-other-lb"}, +] + + +# ---- HAR ---- +def test_har_maps_method_path_query_and_body(): + recs, _ = traffic.from_har(HAR) + assert len(recs) == 2 + r = recs[0] + assert r.method == "POST" and r.path == "/api/pay" + assert r.query == {"ref": ["abc", "def"]} # repeated params keep both values + assert r.json_body == {"amount": -5, "to": "x"} # HAR carries bodies; XC logs do not + assert r.user_agent == "curl/8.1" and r.ts == "2026-07-27T10:00:00Z" + assert r.status == 200 and r.source == "har" + + +def test_har_entry_without_body_or_headers_still_parses(): + assert traffic.from_har(HAR)[0][1].path == "/health" + + +# ---- XC access logs ---- +def test_xc_log_maps_the_real_field_names(): + recs, _ = traffic.from_xc_logs(XC_LOGS) + r = recs[0] + assert r.method == "GET" and r.path == "/api/accounts" and r.query == {"id": ["7"]} + assert r.status == 200 and r.user_agent == "node" and r.source == "xc" + assert r.json_body is None # XC captures no request body + + +def test_xc_log_headers_arrive_as_a_json_string(): + assert traffic.from_xc_logs(XC_LOGS)[0][0].headers.get("accept") == "*/*" + + +def test_xc_logs_filter_by_lb_via_vh_name(): + recs, _ = traffic.from_xc_logs(XC_LOGS, lb="nimbus-www") + assert len(recs) == 1 and recs[0].path == "/api/accounts" + + +# ---- redaction: a correctness requirement, because the sample is distributed ---- +def test_authorization_and_cookie_are_stripped_at_ingest(): + recs, red = traffic.from_har(HAR) + assert "Authorization" not in recs[0].headers and "authorization" not in recs[0].headers + assert "sk-live-SECRET" not in json.dumps(recs[0].model_dump()) + assert red.get("authorization") == 1 + + +def test_cookie_stripped_from_xc_records_too(): + recs, red = traffic.from_xc_logs(XC_LOGS) + assert "cookie" not in {k.lower() for k in recs[0].headers} + assert red.get("cookie") == 1 + + +def test_secret_looking_body_fields_are_redacted_not_dropped(): + har = {"log": {"entries": [{"request": { + "method": "POST", "url": "https://x/api/login", "headers": [], + "postData": {"mimeType": "application/json", + "text": '{"username": "jo", "password": "hunter2", "token": "t-1"}'}}}]}} + r, red = traffic.from_har(har) + body = r[0].json_body + assert body["username"] == "jo" # shape preserved so matchers still work + assert body["password"] == traffic.REDACTED and body["token"] == traffic.REDACTED + assert red.get("password") == 1 and red.get("token") == 1 + + +def test_extra_redact_patterns_are_configurable(): + har = {"log": {"entries": [{"request": { + "method": "GET", "url": "https://x/a", "headers": [{"name": "X-Trace", "value": "abc"}]}}]}} + recs, _ = traffic.from_har(har, redact_headers=("x-trace",)) + assert "X-Trace" not in recs[0].headers + + +def test_src_ip_is_not_carried_into_the_record(): + """The bundle ships the sample; a client IP is personal data and nothing in G2 needs it.""" + assert "3.21.170.57" not in json.dumps([r.model_dump() for r in traffic.from_xc_logs(XC_LOGS)[0]]) + + +# ---- robustness: a malformed line must never kill a replay ---- +def test_malformed_records_are_counted_not_raised(tmp_path): + p = tmp_path / "s.jsonl" + p.write_text('{"method":"GET","path":"/ok"}\nnot json at all\n{"no_path":true}\n') + recs, red = traffic.from_jsonl(p.read_text()) + assert len(recs) == 1 and recs[0].path == "/ok" + assert red.get("_skipped") == 2 + + +def test_load_dispatches_on_extension(tmp_path): + (tmp_path / "t.har").write_text(json.dumps(HAR)) + (tmp_path / "t.jsonl").write_text('{"method":"GET","path":"/a"}\n') + assert len(traffic.load(str(tmp_path / "t.har"))[0]) == 2 + assert len(traffic.load(str(tmp_path / "t.jsonl"))[0]) == 1 + with pytest.raises(ValueError): + traffic.load(str(tmp_path / "t.txt")) + + +def test_empty_source_is_empty_not_an_error(): + assert traffic.from_har({"log": {"entries": []}}) == ([], {}) + assert traffic.from_jsonl("") == ([], {}) + + +def test_record_round_trips_through_a_probe_request_shape(): + """A finding's own exploit is a valid record — that is what makes the acceptance testable + without inventing a second fixture for the exploit side.""" + rec = traffic.from_probe_request({"method": "POST", "path": "/api/pay", + "json_body": {"amount": -1}}, source="probe") + assert rec.method == "POST" and rec.path == "/api/pay" and rec.json_body == {"amount": -1} + + +def test_the_committed_nimbus_sample_ingests_and_holds_one_exploit(): + """G2's acceptance leans on this fixture: one negative-amount transfer among legitimate ones.""" + from pathlib import Path + recs, red = traffic.load(str(Path(__file__).parent.parent / "bench/fixtures/traffic/nimbus-sample.jsonl")) + assert len(recs) == 200 and red.get("_skipped") is None + neg = [r for r in recs if r.json_body and (r.json_body.get("amount") or 0) < 0] + pays = [r for r in recs if r.path == "/api/pay"] + assert len(neg) == 1 and len(pays) == 61 # exactly one exploit, 60 legitimate transfers