From 922869905483ef7e9218e3a6433de96746623b2d Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 06:43:59 +0100 Subject: [PATCH 01/22] docs(auth): split AUTH-05 evidence foundation --- .agent-loop/LOOP_STATE.md | 26 +-- .agent-loop/REVIEW_LOG.md | 16 ++ .agent-loop/WORK_QUEUE.md | 9 +- .../CHUNK_MAP.md | 18 +- .../DISCOVERY.md | 18 ++ .../PLAN.md | 3 +- .../STATUS.md | 26 ++- ...TH-001-05-authority-evidence-foundation.md | 133 +++--------- ...001-05A-shared-audit-authority-evidence.md | 185 +++++++++++++++++ ...S-AUTH-001-05B-idempotency-invalidation.md | 189 ++++++++++++++++++ 10 files changed, 483 insertions(+), 140 deletions(-) create mode 100644 .agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md create mode 100644 .agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05B-idempotency-invalidation.md diff --git a/.agent-loop/LOOP_STATE.md b/.agent-loop/LOOP_STATE.md index 4cf2f833..178615f9 100644 --- a/.agent-loop/LOOP_STATE.md +++ b/.agent-loop/LOOP_STATE.md @@ -3,21 +3,23 @@ ## Current State - Active initiative: `WS-AUTH-001` - Workstream Authorization Service -- Active planning chunk: none +- Active planning chunk: `WS-AUTH-001-05A` - Shared Audit Ownership And + Append-Only Authority Evidence - Active implementation chunk: none -- Branch: `codex/ws-auth-001-04b-post-merge-memory` -- Worktree: `/home/abiorh/flow/workstream-authorization-service` -- Status: AUTH-04B merged through PR #113 as `05a63c8` after Backend, Agent - Gates, CodeRabbit, required internal review, and explicit human approval - passed. +- Branch: `codex/ws-auth-001-05-authority-evidence` +- Worktree: `/home/abiorh/flow/workstream-auth-001-05` +- Status: AUTH-04B post-merge memory merged through PR #114 as `97cd0f5`. + The user explicitly started AUTH-05. Required L1 plan review rejected the + combined contract before runtime edits and required the 05A/05B split. - Prior `WS-AUTH-001-01` reviewed implementation SHA: `be0b836` - Prior `WS-AUTH-001-01` final merged branch head: `b5217e1` -- Latest integrated `main` merge commit: `05a63c8` -- Current gate: publish and merge the AUTH-04B post-merge memory update, then - stop. -- Size checkpoint: implementation is 480 changed non-comment production lines; - the required 350-line inspection passed and the 500-line hard stop holds. -- Next chunk: none; do not start `WS-AUTH-001-05` automatically. +- Latest integrated `main` merge commit: `97cd0f5` +- Current gate: repair and re-review the AUTH-05A and AUTH-05B contracts before + any runtime edit. +- Size checkpoint: each child has a 350-line inspection and a 500-line + production hard stop. +- Next chunk: AUTH-05A only after its repaired preimplementation review passes. + AUTH-05B and AUTH-06 remain inactive. - Focused evidence: 77 isolated rate/config tests pass at 97 percent subsystem coverage, 59 final config/object-graph tests pass, and the exact migration proof and real API E2E pass. Final GitHub Backend passed 937 tests at 82.15 diff --git a/.agent-loop/REVIEW_LOG.md b/.agent-loop/REVIEW_LOG.md index 24a40adc..e00bd432 100644 --- a/.agent-loop/REVIEW_LOG.md +++ b/.agent-loop/REVIEW_LOG.md @@ -1,5 +1,21 @@ # Review Log +## 2026-07-14 - WS-AUTH-001-05 Plan Review Split Required + +After AUTH-04B post-merge memory merged through PR #114 as `97cd0f5`, the user +explicitly started AUTH-05. Required L1 engineering, QA/CI, and +security/privacy/product plan review rejected the combined contract before any +runtime edit. Migration `0017` is already owned by AUTH-04B, dedicated audit and +authorization tests were absent, and shared audit custody plus idempotency and +invalidation were not credibly reviewable as one sub-500-line production diff. + +The repaired plan splits AUTH-05 into 05A shared audit ownership/append-only +authority evidence on migration `0018`, then 05B idempotency/invalidation on +`0019`. Both retain caller-owned transactions, focused 90 percent subsystem +coverage, the global 78 percent CI floor, full reviewer fanout, and separate +human merge/start gates. No runtime implementation is active pending repaired +contract review. + ## 2026-07-14 - WS-AUTH-001-04B Merged PR #113 merged `WS-AUTH-001-04B` to `main` as `05a63c8` after final branch head diff --git a/.agent-loop/WORK_QUEUE.md b/.agent-loop/WORK_QUEUE.md index 740ca009..6714066d 100644 --- a/.agent-loop/WORK_QUEUE.md +++ b/.agent-loop/WORK_QUEUE.md @@ -4,7 +4,7 @@ | Chunk | Title | Risk | Status | |---|---|---:|---| -| - | No active implementation chunk | - | AUTH-04B post-merge memory in progress; no product implementation active | +| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | Contract repair and preimplementation re-review; no runtime implementation active | ## Planned Next @@ -57,9 +57,10 @@ ## Proposed Next -`WS-AUTH-001-04B` merged through PR #113 as `05a63c8` after all required checks -and reviews passed. Its post-merge memory update is active; no AUTH product -implementation is active. Do not start AUTH-05 or POL-002-04 automatically. +AUTH-04B post-merge memory merged through PR #114 as `97cd0f5`. The user +explicitly started AUTH-05. Required plan review rejected the combined contract +before runtime edits and required children 05A/05B. Only 05A contract repair is +active. Do not implement 05B, AUTH-06, or POL-002-04 automatically. Coverage R10 merged through PR #108. Do not start 01B2, chunk 02, or another coverage implementation chunk from this worktree. diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md index 5f15136e..288a4d3a 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md @@ -18,7 +18,9 @@ stopped. | `WS-AUTH-001-04` | Request, Error, And API Control Foundation | L1 | Split before implementation into 04A and 04B | | `WS-AUTH-001-04A` | Request And Error Context | L1 | Merged through PR #111 as `90c9a28` | | `WS-AUTH-001-04B` | PostgreSQL Rate Controls | L1 | Merged through PR #113 as `05a63c8` | -| `WS-AUTH-001-05` | Authority Evidence And Idempotency Foundation | L1 | Proposed | +| `WS-AUTH-001-05` | Authority Evidence And Idempotency Foundation | L1 | Split before implementation into 05A and 05B | +| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | Contract repair and preimplementation re-review | +| `WS-AUTH-001-05B` | Authority Idempotency And Invalidation Foundation | L1 | Inactive pending 05A merge/memory and separate explicit start | | `WS-AUTH-001-06` | Canonical Actor Profile And Identity Link | L1 | Proposed | | `WS-AUTH-001-07` | Authorization Kernel And Permission Registry | L1 | Proposed | | `WS-AUTH-001-08` | Bootstrap And Administrative Role Grants | L1 | Proposed | @@ -40,7 +42,8 @@ WS-AUTH-001-PLAN -> WS-AUTH-001-03 -> WS-AUTH-001-04A -> WS-AUTH-001-04B --> WS-AUTH-001-05 +-> WS-AUTH-001-05A +-> WS-AUTH-001-05B -> WS-AUTH-001-06 -> WS-AUTH-001-07 -> WS-AUTH-001-08 @@ -61,7 +64,9 @@ WS-AUTH-001-PLAN - Parent chunk 04 was split before implementation. Chunk 04A establishes request/correlation and additive error compatibility; chunk 04B later owns durable PostgreSQL rate controls and its migration. -- Chunk 05 evolves the shared audit path and idempotency before mutations. +- Parent chunk 05 was split before implementation. Chunk 05A owns shared audit + schema/writer custody and append-only authority evidence; chunk 05B owns + idempotency and typed invalidation orchestration. - Chunk 06 establishes canonical actor resolution while preserving only the enumerated non-authoritative legacy workflow-eligibility consumers required for intermediate-release operability. @@ -89,5 +94,8 @@ post-merge memory merged through PR #112 as `7749f54`. The user explicitly started AUTH-04B. Its repaired contract passed at `b5dceb1`; bounded implementation and all required internal review tracks passed, and PR #113 merged as `05a63c8` after Backend, Agent Gates, CodeRabbit, and explicit human -approval passed. AUTH-04B post-merge memory is the current gate. Stop; do not -start AUTH-05 or POL-002-04. +approval passed. AUTH-04B post-merge memory then merged through PR #114 as +`97cd0f5`, and the user explicitly started AUTH-05. Required plan review +rejected the combined contract before runtime edits and required 05A/05B. +Repaired contract review is the current gate. Do not implement either child +until its own review passes, and do not start AUTH-06 or POL-002-04. diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/DISCOVERY.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/DISCOVERY.md index b4454949..99e56f08 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/DISCOVERY.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/DISCOVERY.md @@ -157,6 +157,24 @@ Production deployment inputs remain externally supplied: - Fixed internal system actors for server-owned automation. - One PR-sized chunk at a time with deterministic evidence and internal review. +## AUTH-05 Delta Discovery (2026-07-14) + +- Current migration head is `0017_api_controls`; the original AUTH-05 + `0017_*` allowance collides and cannot be implemented. +- `AuditEvent` remains in `tasks.models`. `AuditRepository` owns a shared insert + implementation, while `TaskRepository` still duplicates insert/read logic + used by task/checker services. +- Legacy audit rows require non-null external issuer/subject, token-role, and + claim fields. Authority events need a conditional privacy-safe envelope that + preserves legacy rows without copying those identity-provider fields. +- The combined shared-audit schema/writer, append-only database custody, + idempotency state machine, and concurrency proof cross two persistent + subsystem boundaries. Required L1 plan review split them into 05A (`0018`) + and 05B (`0019`) before runtime implementation. +- D13 provider-neutral verifier adoption does not block either child and remains + a separate reviewed chunk after the shared external-service adapter + foundation exists. + ## AUTH-04 Delta Discovery (2026-07-13) ### Current behavior diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/PLAN.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/PLAN.md index 6a8ab1a1..625a27bb 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/PLAN.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/PLAN.md @@ -130,7 +130,8 @@ proving the same token role alone no longer authorizes. evidence workflow. 4. Establish request/correlation context, structured errors, and API rate controls. -5. Evolve shared audit evidence and add canonical idempotency/invalidation. +5. Evolve shared audit evidence in 05A, then add canonical + idempotency/invalidation in 05B. 6. Migrate to canonical profile/link semantics and first-human resolution. 7. Implement the minimal registered permission and AuthorizationService kernel before protected authority-management APIs. diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md index 9b959c54..12887c2e 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md @@ -43,11 +43,16 @@ internally approved at final SHA `922778b`; reviewed production SHA is `67484b5`. The Backend CI isolation repair passed exact review at `4fb846c`. Final branch head `94fb2fe` passed Backend, Agent Gates, and CodeRabbit, then explicit human approval merged PR #113 to `main` as `05a63c8` on 2026-07-14. -AUTH-04B post-merge memory is active; AUTH-05 remains inactive. +AUTH-04B post-merge memory merged through PR #114 as `97cd0f5`. The user then +explicitly started AUTH-05. Required L1 plan review rejected the combined +audit/idempotency contract before runtime edits and required children 05A and +05B. Contract repair and re-review are active; no runtime implementation is +active. ## Active planning chunk -None. +`WS-AUTH-001-05A` - Shared Audit Ownership And Append-Only Authority Evidence. +Contract repair and required preimplementation re-review only. ## Active implementation chunk @@ -55,8 +60,8 @@ None. ## Current implementation branch -None. Post-merge memory uses `codex/ws-auth-001-04b-post-merge-memory` and makes -no product implementation change. +`codex/ws-auth-001-05-authority-evidence` in +`/home/abiorh/flow/workstream-auth-001-05`. ## Chunk status @@ -69,7 +74,9 @@ no product implementation change. | `WS-AUTH-001-04` | Split | `codex/ws-auth-001-04-request-api-controls` | - | Parent split before runtime implementation. | | `WS-AUTH-001-04A` | Merged | `codex/ws-auth-001-04-request-api-controls` | #111 | Merged as `90c9a28`; production review `cdcaf77`; final branch head `36c4aa5`. | | `WS-AUTH-001-04B` | Merged | `codex/ws-auth-001-04b-postgres-rate-controls` | #113 | Merged as `05a63c8`; production review `67484b5`; final branch head `94fb2fe`. | -| `WS-AUTH-001-05` | Proposed | - | - | Authority evidence and idempotency foundation. | +| `WS-AUTH-001-05` | Split | `codex/ws-auth-001-05-authority-evidence` | - | Parent split before implementation into 05A and 05B. | +| `WS-AUTH-001-05A` | Contract review | `codex/ws-auth-001-05-authority-evidence` | - | Shared audit ownership and append-only authority evidence; runtime edits prohibited pending review. | +| `WS-AUTH-001-05B` | Inactive | - | - | Idempotency and invalidation; requires 05A merge/memory and separate explicit start. | | `WS-AUTH-001-06` | Proposed | - | - | Canonical actor profile and identity link. | | `WS-AUTH-001-07` | Proposed | - | - | Authorization kernel and permissions. | | `WS-AUTH-001-08` | Proposed | - | - | Bootstrap and administrative grants. | @@ -84,10 +91,11 @@ no product implementation change. ## Blockers -No active implementation blocker because no AUTH product chunk is active. -AUTH-05 may start only after this memory update merges and the user gives a -separate explicit start signal. AUTH-04B owns migration `0017`, following the -now-owned `0016` prefix on current main. Non-test +AUTH-05A runtime implementation is blocked on repaired-contract review. The +combined AUTH-05 contract was rejected before code changes because migration +`0017` is already owned by AUTH-04B and the shared-audit plus idempotency scope +was not reviewable as one L1 change. AUTH-05A now owns proposed migration +`0018`; AUTH-05B later owns proposed migration `0019`. Non-test operators must later supply explicit classification evidence rather than inferred kinds before the owning canonical actor migration. diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05-authority-evidence-foundation.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05-authority-evidence-foundation.md index e6547f67..12abd377 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05-authority-evidence-foundation.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05-authority-evidence-foundation.md @@ -1,125 +1,40 @@ # Chunk Contract: WS-AUTH-001-05 - Authority Evidence And Idempotency Foundation +## Status + +Split before runtime implementation after required L1 plan review. No parent +chunk implementation is permitted. + ## Parent initiative `WS-AUTH-001` - Workstream Authorization Service ## Goal -Evolve the existing `audit_events`/`AuditRepository` path with canonical -mutation idempotency and authority invalidation evidence before actor or grant -mutations use it. - -## Why this chunk exists - -Authority-changing APIs cannot ship with temporary provenance that a later -chunk would need to reconstruct. - -## Approved plan reference - -- INTENT: `.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/INTENT.md` -- PLAN: `.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/PLAN.md` -- CHUNK_MAP: `.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md` - -## Risk class - -L1 - -## SLA - -P1 - -## Allowed files - -```text -backend/app/modules/audit/** -backend/app/modules/authorization/** -backend/app/db/models.py -backend/app/modules/tasks/models.py -backend/app/modules/tasks/repository.py -backend/app/modules/tasks/service.py -backend/alembic/versions/0017_*.py -backend/tests/test_auth.py -backend/tests/test_alembic.py -backend/tests/test_tasks.py -backend/scripts/api_contract_e2e.py -docs/operations_authorization_service.md -.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/** -.agent-loop/LOOP_STATE.md -.agent-loop/WORK_QUEUE.md -.agent-loop/REVIEW_LOG.md -``` - -## Not allowed - -```text -actor/profile migration or first-access behavior -permission/grant evaluation -admin/project grants -project/task/checker authorization cutover -backfilling fabricated authority events -``` - -## Acceptance criteria - -- Idempotency rows store canonical request hash, operation, actor reference, - status, and bounded committed response reference with unique replay guards. -- Authority events have final versioned fields for acting/target references, - matched grant, project, reason, idempotency key, bounded before/after state, - and database time. -- Existing `AuditEvent` is extended; no parallel AuthorityEvent table/model is - created. `TaskRepository.add_audit_event` delegates to or is consolidated - with `AuditRepository` so one shared writer owns persistence. -- Audit persistence exposes insert/read operations only through supported - application paths. Migration-level enforcement rejects authority-event - update/delete by the application database role; focused tests prove attempts - fail without changing prior rows. The runbook defines retention and the - separately controlled privileged database-maintenance path. -- Authorization-owned idempotency models, repository queries, and service - orchestration remain separate feature modules. Middleware, generic core - helpers, and ORM models do not own replay queries or transaction policy. -- The injected AsyncSession remains the UnitOfWork boundary. Caller services - own commit/rollback; no generic parallel UoW wrapper is introduced. -- Session-bound repository APIs let callers atomically persist business state, - idempotency result, authority event, and invalidation event. -- Migration owns idempotency/audit constraints, indexes, UTC/database-time - defaults, prior-head upgrade, downgrade, and re-upgrade. -- Each later authority-mutation chunk must test its specified successful and - denied audit events atomically when that behavior is introduced; chunk 16 - cannot backfill an omitted event. +Establish durable authority evidence before actor or grant mutations use it. -## Verification commands +## Split result -```bash -(cd backend && WORKSTREAM_DATABASE_URL= .venv/bin/alembic upgrade head) -(cd backend && WORKSTREAM_DATABASE_URL= .venv/bin/alembic downgrade -1) -(cd backend && WORKSTREAM_DATABASE_URL= .venv/bin/alembic upgrade head) -(cd backend && .venv/bin/python -m ruff check app tests) -(cd backend && WORKSTREAM_DATABASE_URL= .venv/bin/python -m pytest -q) -(cd backend && WORKSTREAM_DATABASE_URL= .venv/bin/python scripts/api_contract_e2e.py) -python3 scripts/check_stale_workstream_wording.py -python3 scripts/check_markdown_links.py -git diff --check -``` +The original contract combined two persistent subsystem boundaries and was not +credibly reviewable below the L1 size ceiling. Implementation is owned by: -## Required reviewers +- `WS-AUTH-001-05A` - Shared Audit Ownership And Append-Only Authority Evidence +- `WS-AUTH-001-05B` - Authority Idempotency And Invalidation Foundation -- senior engineering -- QA/test -- security/auth -- product/ops -- architecture -- CI integrity -- docs -- reuse/dedup -- test delta +AUTH-05A must merge and complete post-merge memory before AUTH-05B starts. +AUTH-06 must not start until both children merge and their memory is current. -## Human review focus +## Preserved boundaries -Review reuse of AuditEvent/AuditRepository, idempotency uniqueness/replay, -AsyncSession transaction ownership, and unchanged product authorization. +- Existing `AuditEvent` evolves in place; no parallel authority-event table. +- `AuditRepository` becomes the only supported insert/read implementation. +- Authorization owns idempotency and invalidation orchestration. +- The injected `AsyncSession` remains the unit-of-work boundary. +- No route, permission, grant, actor migration, or product cutover is activated. +- D13 provider-neutral verifier adoption remains a separate reviewed chunk + after the shared external-service adapter foundation exists. -## Stop conditions +## Stop condition -Stop if final evidence requires a canonical ActorProfile FK before migration; -use stable actor-reference fields rather than introducing a parallel actor. +Stop at this split record. Implement only one reviewed child contract at a +time. diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md new file mode 100644 index 00000000..1ca831cc --- /dev/null +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md @@ -0,0 +1,185 @@ +# Chunk Contract: WS-AUTH-001-05A - Shared Audit Ownership And Append-Only Authority Evidence + +## Status + +Active for contract repair and required preimplementation review. Runtime edits +remain prohibited until that review passes. + +## Parent initiative + +`WS-AUTH-001` - Workstream Authorization Service + +## Goal + +Evolve the existing `audit_events` path into one privacy-safe, append-only, +versioned authority-evidence envelope while preserving every legacy lifecycle +event and existing task/checker behavior. + +## Risk and circuit breaker + +- Risk: L1 / SLA P1. +- Inspect scope at 350 changed non-comment production lines. +- Hard stop at 500 changed non-comment production lines, counting + `backend/app/**` plus migration code; tests and evidence do not justify + exceeding the production limit. + +## Allowed files + +```text +backend/app/modules/audit/** +backend/app/db/models.py +backend/app/modules/tasks/models.py +backend/app/modules/tasks/repository.py +backend/alembic/versions/0018_authority_audit_evidence.py +backend/tests/test_audit.py +backend/tests/test_alembic.py +backend/tests/test_tasks.py +docs/architecture_data_model.md +docs/operations_authorization_service.md +.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/** +.agent-loop/LOOP_STATE.md +.agent-loop/WORK_QUEUE.md +.agent-loop/REVIEW_LOG.md +``` + +`backend/tests/test_tasks.py` may receive only bounded delegation/regression +proof; the existing full task suite remains an unchanged regression target. + +## Not allowed + +```text +authorization idempotency records or replay orchestration +routes, dependencies, middleware, or generic core helpers +actor/profile migration or first-access behavior +permission definitions/evaluation or admin/project grants +project/task/submission/checker authorization cutover +invalidation consumers, caches, queues, or external delivery +token-role or IdentityIssuerVerifier/factory changes +workflow, dependency, coverage-threshold, skip, or exclusion changes +backfilled or fabricated authority history +repository or service commit/rollback/session creation +``` + +## Authority event envelope + +Migration `0018_authority_audit_evidence` revises `0017_api_controls` and +extends the existing `audit_events` table. It does not create a parallel event +table. Existing rows and legacy writers retain their current values and public +behavior; no legacy row is reclassified as authority evidence. + +New rows use `event_domain = authority`, `event_version = 1`, and a typed event +token from the adopted specification. Authority rows require: + +- database-owned `occurred_at`; +- `request_id` and `correlation_id` UUIDs; +- namespaced acting reference kind (`legacy_actor`, `actor_profile`, or + `system_principal`) plus stable opaque reference; +- optional target actor reference with all-or-none kind/reference presence; +- optional matched-grant, permission, project, resource type/id, and target + grant/link reference; +- optional bounded reason and denial code; +- optional idempotency-record reference; +- optional invalidation cause event and invalidation target kind/id; +- typed shallow before/after fact objects. + +Authority rows never persist external issuer, external subject, token roles, +claim snapshots, request bodies, raw tokens, JWKS documents, secrets, emails, +or URLs. Legacy claim fields are nullable only to permit authority rows and +must be `NULL`, `[]`, or `{}` as appropriate for the authority domain. +Conditional database constraints reject mixed legacy/authority shapes. + +Reference/type/reason tokens have explicit length and character bounds. +Before/after facts are JSON objects with at most 16 allowlisted scalar keys, +no nested containers, and at most 4096 encoded bytes per object. The typed +writer applies per-event allowed-key sets; database checks enforce object type +and byte bounds. + +## Shared writer and append-only custody + +- `AuditRepository` is the only supported insert/read implementation. +- `TaskRepository.add_audit_event` and `list_audit_events` remain temporary + compatibility methods but delegate the same session/event/query to + `AuditRepository`; task/checker call sites and responses do not change. +- Supported application APIs expose insert/read only. +- Named PostgreSQL triggers reject every `audit_events` UPDATE, DELETE, and + TRUNCATE, regardless of event domain or session flag. Failed attempts leave + both legacy and authority rows unchanged. +- No application-settable GUC or current-user bypass exists. +- The protection is a normal-DML custody boundary, not a defense against the + table owner or DDL credentials. Production rollout requires a distinct + non-owner runtime role without trigger/DDL privileges. The runbook documents + that deployment gate and a separately controlled DB-owner maintenance + procedure using an exclusive lock, explicit trigger disable/re-enable, + transaction, change record, and post-check. Migration teardown is the only + ordinary trigger-removal path. + +## Migration custody + +- Upgrade `0017 -> 0018` preserves representative populated prior-head domain + and legacy audit rows without fabricating authority fields. +- Tests assert exact columns, nullability, defaults, constraints, indexes, + trigger/function names, invalid direct inserts, and database time. +- Downgrade takes writer-blocking locks before inspection and refuses without + mutation when any authority-domain audit row exists. +- Privileged test cleanup may delete only explicit fixture authority rows after + deliberately disabling/re-enabling the trigger under lock; legacy rows are + never deleted by downgrade. +- Empty authority evidence permits `0018 -> 0017 -> 0018`; re-upgrade recreates + all triggers. Destructive migration tests restore Alembic `head` in `finally`. + +## Acceptance criteria + +- Legacy audit writers/readers and task/checker response projections remain + behaviorally unchanged. +- Typed allowed, denied, and invalidation authority event shapes persist only + bounded non-sensitive evidence and reject malformed/mixed shapes. +- Application-role normal DML cannot update, delete, or truncate any audit row. +- Shared-writer tests instrument `AuditRepository` and prove TaskRepository + compatibility methods pass the same session/event and return the shared + result; row-count-only proof is insufficient. +- No new route, dependency, middleware, permission, grant, actor, product + authority behavior, or invalidation consumer exists. + +## Verification + +```bash +(cd backend && .venv/bin/python -m ruff check app tests) +(cd backend && isolated PostgreSQL: pytest -q tests/test_alembic.py::<0018-proof> tests/test_audit.py tests/test_tasks.py::) +(cd backend && focused AUTH-05A coverage >= 90 percent) +(cd backend && isolated full suite --cov=app --cov-fail-under=78) +(cd backend && .venv/bin/docstr-coverage --config .docstr.yaml) +python3 scripts/check_stale_workstream_wording.py +python3 scripts/check_stale_authorization_docs.py +python3 scripts/check_stale_artifact_contracts.py +python3 scripts/check_markdown_links.py +python3 scripts/check_internal_review_evidence.py +git diff --check +``` + +Run the exact migration node before audit/delegation tests on the same isolated +database. Test delta must be additive: no assertion, raises, skip, xfail, +threshold, workflow, or exclusion weakening. + +## Required reviewers + +- senior engineering +- QA/test +- security/auth and privacy +- product/ops +- architecture +- CI integrity +- docs +- reuse/dedup +- test delta + +## Human review focus + +Review legacy audit compatibility, sole writer ownership, authority-envelope +privacy, unconditional normal-DML immutability, migration custody, and the +documented non-owner production role requirement. + +## Stop conditions + +Stop if legacy event behavior changes, authority evidence needs a canonical +ActorProfile FK, normal application DML can mutate history, production scope +exceeds the circuit breaker, or tests/CI must be weakened. diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05B-idempotency-invalidation.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05B-idempotency-invalidation.md new file mode 100644 index 00000000..17654d70 --- /dev/null +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05B-idempotency-invalidation.md @@ -0,0 +1,189 @@ +# Chunk Contract: WS-AUTH-001-05B - Authority Idempotency And Invalidation Foundation + +## Status + +Inactive until AUTH-05A merges, its post-merge memory is current, and the user +gives a separate explicit start signal. + +## Parent initiative + +`WS-AUTH-001` - Workstream Authorization Service + +## Goal + +Add concurrency-safe authority-mutation reservation/replay and typed success, +denial, and invalidation orchestration on the shared AUTH-05A audit envelope. + +## Risk and circuit breaker + +- Risk: L1 / SLA P1. +- Inspect scope at 350 changed non-comment production lines. +- Hard stop at 500 changed non-comment production lines, counting + `backend/app/**` plus migration code. + +## Allowed files + +```text +backend/app/modules/authorization/** +backend/app/modules/audit/** +backend/app/db/models.py +backend/alembic/versions/0019_authority_idempotency.py +backend/tests/test_authorization.py +backend/tests/test_audit.py +backend/tests/test_alembic.py +docs/architecture_data_model.md +docs/operations_authorization_service.md +.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/** +.agent-loop/LOOP_STATE.md +.agent-loop/WORK_QUEUE.md +.agent-loop/REVIEW_LOG.md +``` + +## Not allowed + +```text +routes, dependencies, middleware, or generic unit-of-work wrappers +actor/profile migration or first-access behavior +permissions, grant tables/APIs, or product authorization cutover +invalidation consumers, caches, queues, or external delivery +token-role or IdentityIssuerVerifier/factory changes +raw request/response bodies, URLs, PII, secrets, or arbitrary JSON persistence +workflow, dependency, threshold, skip, or exclusion changes +repository/service commit, rollback, or parallel session creation +``` + +## Idempotency schema and namespace + +Migration `0019_authority_idempotency` revises AUTH-05A's `0018` migration and +creates `authority_idempotency_records`. Each record contains: + +- UUID primary key and canonical UUID idempotency key; +- actor reference kind plus stable opaque actor reference; +- closed operation token; +- canonical request digest using existing + `app.core.hashing.canonical_json_hash` (`sha256:` plus 64 lowercase hex); +- status `pending` or `committed`; +- typed committed response reference: resource type, resource ID, optional + version, and successful HTTP status; never a body, URL, or free-form JSON; +- database-owned created/committed timestamps. + +The unique replay namespace is +`(actor_ref_kind, actor_ref, operation, idempotency_key)`. Different actors or +reference kinds may reuse a key without collision or visibility. Operation, +target project/resource, and mutation payload are included in the canonical +request object/digest. Request ID, correlation ID, and the idempotency key are +excluded from the digest so legitimate retries may carry new request context. +The raw request is never persisted. + +The canonical request object is a typed, bounded JSON-compatible dictionary. +It uses the shared canonical hash helper; no authorization-local encoder is +introduced. + +## State machine and transaction ownership + +- Reservation runs before any business-state flush. +- `INSERT ... ON CONFLICT` plus a locked read serializes concurrent users of + one replay namespace. +- A new reservation returns `claimed` with a `pending` row in the caller's + transaction. +- Same namespace and digest after commit returns `replay` with the existing + typed committed reference and does not append duplicate success/invalidation + events. +- Same namespace with a different digest returns a stable `mismatch` outcome; + caller-owned application orchestration records the required denial event in + a clean transaction before surfacing `idempotency_mismatch`. +- Completion atomically changes only the caller's pending row to `committed` + and writes its typed response reference. +- A deferred database constraint rejects commit while a record is still + pending. Rollback/cancellation removes the reservation and all co-transaction + business/audit/invalidation rows; retry may then claim normally. A pending + row is never stolen, expired, or treated as success. +- Repository/service methods may insert, lock, flush, and return typed outcomes + only. The injected caller session alone commits or rolls back; no parallel + unit-of-work abstraction or session is introduced. + +## Invalidation and denial evidence + +Invalidation is an `AuthorityInvalidationRequested` authority-domain +`AuditEvent`, not a second event table. Authorization owns typed orchestration; +AuditRepository remains the persistence owner. The event links its causing +authority event, exact target kind/reference, request/correlation IDs, actor, +reason, and idempotency record when applicable. AUTH-05B introduces persistence +and orchestration only: no cache, worker, adapter, or authority behavior consumes +the event. + +Allowed, denied, and invalidation events use stable tokens. Denial/mismatch +orchestration returns a typed result rather than committing or raising inside +the repository; the caller commits the denial evidence before translating the +result to an API error. Denials are not stored as successful replay results. + +## Migration custody + +- Upgrade preserves all AUTH-05A audit rows and creates no idempotency or + invalidation evidence. +- Tests assert exact table columns, constraints, indexes, deferred trigger, + defaults, invalid rows, and database time. +- Downgrade takes writer-blocking locks and refuses without mutation when any + idempotency row or audit row referencing idempotency/invalidation evidence + exists. Legacy and AUTH-05A audit rows remain intact. +- Privileged fixture cleanup permits `0019 -> 0018 -> 0019`; re-upgrade restores + the pending-commit guard. Destructive tests restore `head` in `finally`. + +## Acceptance criteria + +- Independent-session exact concurrent retries produce one committed result + and one success/invalidation event set. +- Concurrent mismatched requests produce one winner and one privacy-safe + mismatch outcome; different actors/kinds remain isolated. +- Injected failure after reservation, synthetic business flush, audit insert, + invalidation insert, completion, and commit attempt leaves either the complete + transaction or nothing. Repositories never commit/rollback. +- Retry after rollback claims normally; commit of unfinished pending state + fails closed and leaves no durable pending row after caller rollback. +- Exact replay never duplicates business, success, or invalidation evidence. +- No route, permission, grant, actor, product authority behavior, or + invalidation consumer exists. + +## Verification + +```bash +(cd backend && .venv/bin/python -m ruff check app tests) +(cd backend && isolated PostgreSQL: pytest -q tests/test_alembic.py::<0019-proof> tests/test_authorization.py tests/test_audit.py) +(cd backend && focused AUTH-05B coverage >= 90 percent) +(cd backend && isolated full suite --cov=app --cov-fail-under=78) +(cd backend && .venv/bin/docstr-coverage --config .docstr.yaml) +python3 scripts/check_stale_workstream_wording.py +python3 scripts/check_stale_authorization_docs.py +python3 scripts/check_stale_artifact_contracts.py +python3 scripts/check_markdown_links.py +python3 scripts/check_internal_review_evidence.py +git diff --check +``` + +Run the exact migration node before audit/authorization tests on the same +isolated database. Test delta must be additive and all CI integrity controls +remain unchanged. + +## Required reviewers + +- senior engineering +- QA/test +- security/auth and privacy +- product/ops +- architecture +- CI integrity +- docs +- reuse/dedup +- test delta + +## Human review focus + +Review replay namespace, canonical hashing, pending-commit prevention, +concurrency/rollback behavior, typed result privacy, invalidation causation, +and caller-owned transaction boundaries. + +## Stop conditions + +Stop if exact concurrency cannot serialize without a committed pending row, +denial evidence requires committing unrelated work, a route/grant/permission +must be introduced, or tests/CI must be weakened. From f4d9035dc139e927d0526e59ec17056851010733 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 06:50:53 +0100 Subject: [PATCH 02/22] docs(auth): lock AUTH-05 child contracts --- ...001-05A-shared-audit-authority-evidence.md | 63 ++++++++++++++++--- ...S-AUTH-001-05B-idempotency-invalidation.md | 11 ++-- 2 files changed, 62 insertions(+), 12 deletions(-) diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md index 1ca831cc..1919b94d 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md @@ -84,9 +84,27 @@ token from the adopted specification. Authority rows require: Authority rows never persist external issuer, external subject, token roles, claim snapshots, request bodies, raw tokens, JWKS documents, secrets, emails, -or URLs. Legacy claim fields are nullable only to permit authority rows and -must be `NULL`, `[]`, or `{}` as appropriate for the authority domain. -Conditional database constraints reject mixed legacy/authority shapes. +or URLs. Conditional database constraints reject mixed legacy/authority +shapes using this exact compatibility matrix: + +- existing `id` remains the event ID, `event_type` remains the typed event + token, and `created_at` remains the legacy timestamp; +- existing `actor_id` stores the stable local acting reference and the new + actor-reference-kind field supplies its namespace; +- existing `entity_type`/`entity_id` remain the primary audited aggregate + reference; optional project/resource fields provide additional exact scope; +- existing `reason` may carry only the bounded authority reason; +- `from_status` and `to_status` are `NULL` because typed before/after facts own + authority state transitions; +- `external_subject` and `external_issuer` are `NULL`; +- `actor_roles = '[]'`, `claim_snapshot = '{}'`, + `auth_source = 'local_authority'`, and `is_dev_auth = false`; +- `event_payload = '{}'`; authority evidence cannot use the legacy JSON payload + as an unbounded escape hatch. + +Only the external subject/issuer columns become nullable for the conditional +authority shape. Existing legacy rows and writers retain all current values +and non-authority requirements. Reference/type/reason tokens have explicit length and character bounds. Before/after facts are JSON objects with at most 16 allowlisted scalar keys, @@ -94,6 +112,20 @@ no nested containers, and at most 4096 encoded bytes per object. The typed writer applies per-event allowed-key sets; database checks enforce object type and byte bounds. +AUTH-05A behavior tests exercise these exact foundation shapes: + +- `SensitiveAuthorizationAllowed` requires permission and forbids denial code, + invalidation cause/target, and idempotency reference; +- `SensitiveAuthorizationDenied` requires permission plus a bounded denial + code and forbids invalidation cause/target and idempotency reference; +- `AuthorityInvalidationRequested` requires cause-event ID plus all-or-none + target kind/reference, forbids denial code, and reserves a `NULL` + idempotency reference until AUTH-05B. + +Resource type/ID, target actor kind/reference, and invalidation target +kind/reference are each all-or-none pairs in typed validation and database +constraints. + ## Shared writer and append-only custody - `AuditRepository` is the only supported insert/read implementation. @@ -143,16 +175,31 @@ and byte bounds. ## Verification ```bash -(cd backend && .venv/bin/python -m ruff check app tests) -(cd backend && isolated PostgreSQL: pytest -q tests/test_alembic.py::<0018-proof> tests/test_audit.py tests/test_tasks.py::) -(cd backend && focused AUTH-05A coverage >= 90 percent) -(cd backend && isolated full suite --cov=app --cov-fail-under=78) -(cd backend && .venv/bin/docstr-coverage --config .docstr.yaml) +cd backend +tmp_dir=$(mktemp -d) +trap 'rm -rf "$tmp_dir"' EXIT +.venv/bin/python scripts/run_isolated_tests.py --metadata-json "$tmp_dir/05a.json" --timeout-seconds 1800 -- \ + .venv/bin/python -m pytest -q \ + tests/test_alembic.py::test_authority_audit_schema_preserves_legacy_and_guards_downgrade \ + tests/test_audit.py \ + tests/test_tasks.py::test_task_repository_delegates_audit_persistence +.venv/bin/python scripts/run_isolated_tests.py --metadata-json "$tmp_dir/05a-coverage.json" --timeout-seconds 1800 -- \ + .venv/bin/python -m pytest -q tests/test_audit.py \ + tests/test_tasks.py::test_task_repository_delegates_audit_persistence \ + --cov=app.modules.audit --cov-report=term-missing --cov-fail-under=90 +.venv/bin/python scripts/run_isolated_tests.py --metadata-json "$tmp_dir/full.json" --timeout-seconds 12600 -- \ + .venv/bin/python -m pytest -q --ignore=tests/test_isolated_database_runner.py \ + --cov=app --cov-report=term-missing --cov-fail-under=78 +.venv/bin/ruff check app tests +.venv/bin/docstr-coverage --config .docstr.yaml +cd .. python3 scripts/check_stale_workstream_wording.py python3 scripts/check_stale_authorization_docs.py python3 scripts/check_stale_artifact_contracts.py python3 scripts/check_markdown_links.py python3 scripts/check_internal_review_evidence.py +if git diff --unified=0 origin/main...HEAD -- backend/tests | \ + rg '^-(.*assert|.*pytest\.raises|.*pytest\.mark\.(skip|xfail)|.*skipTest)'; then exit 1; fi git diff --check ``` diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05B-idempotency-invalidation.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05B-idempotency-invalidation.md index 17654d70..e77fddbe 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05B-idempotency-invalidation.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05B-idempotency-invalidation.md @@ -25,7 +25,6 @@ denial, and invalidation orchestration on the shared AUTH-05A audit envelope. ```text backend/app/modules/authorization/** -backend/app/modules/audit/** backend/app/db/models.py backend/alembic/versions/0019_authority_idempotency.py backend/tests/test_authorization.py @@ -54,8 +53,9 @@ repository/service commit, rollback, or parallel session creation ## Idempotency schema and namespace -Migration `0019_authority_idempotency` revises AUTH-05A's `0018` migration and -creates `authority_idempotency_records`. Each record contains: +Migration `0019_authority_idempotency` uses exact +`down_revision = "0018_authority_audit_evidence"` and creates +`authority_idempotency_records`. Each record contains: - UUID primary key and canonical UUID idempotency key; - actor reference kind plus stable opaque actor reference; @@ -186,4 +186,7 @@ and caller-owned transaction boundaries. Stop if exact concurrency cannot serialize without a committed pending row, denial evidence requires committing unrelated work, a route/grant/permission -must be introduced, or tests/CI must be weakened. +must be introduced, or tests/CI must be weakened. AUTH-05B consumes the merged +05A audit envelope and sole-writer contract without modifying its schema, +triggers, constraints, repository, or writer. A missing audit primitive requires +stop and replan rather than reopening 05A custody. From 7a9023ba88e437bd4e49d6150ff5ff5a5d83c31f Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 06:53:37 +0100 Subject: [PATCH 03/22] docs(auth): finalize AUTH-05A replay reference --- ...TH-001-05A-shared-audit-authority-evidence.md | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md index 1919b94d..ee08c226 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md @@ -114,13 +114,21 @@ and byte bounds. AUTH-05A behavior tests exercise these exact foundation shapes: -- `SensitiveAuthorizationAllowed` requires permission and forbids denial code, - invalidation cause/target, and idempotency reference; +- `SensitiveAuthorizationAllowed` requires permission, forbids denial code and + invalidation cause/target, and permits an optional canonical UUID + idempotency reference for later AUTH-05B orchestration; - `SensitiveAuthorizationDenied` requires permission plus a bounded denial code and forbids invalidation cause/target and idempotency reference; - `AuthorityInvalidationRequested` requires cause-event ID plus all-or-none - target kind/reference, forbids denial code, and reserves a `NULL` - idempotency reference until AUTH-05B. + target kind/reference, forbids denial code, and permits an optional canonical + UUID idempotency reference for later AUTH-05B orchestration. + +The 05A envelope and database constraints are final for this field: no foreign +key exists before AUTH-05B, and 05A introduces no lookup or replay behavior. +Tests persist both `NULL` foundation use and a syntactically valid non-NULL +future reference without creating an idempotency record. AUTH-05B consumes this +existing field without changing audit schema, constraints, repository, or +writer behavior. Resource type/ID, target actor kind/reference, and invalidation target kind/reference are each all-or-none pairs in typed validation and database From 5f9a49f46cebb678e309eb15b3d1d8d4bd163f76 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 10:59:54 +0100 Subject: [PATCH 04/22] feat(auth): add append-only authority audit evidence --- .agent-loop/LOOP_STATE.md | 17 +- .agent-loop/REVIEW_LOG.md | 11 + .agent-loop/WORK_QUEUE.md | 2 +- .../CHUNK_MAP.md | 2 +- .../STATUS.md | 21 +- ...001-05A-shared-audit-authority-evidence.md | 4 +- .../versions/0018_authority_audit_evidence.py | 232 ++++++++++++ backend/app/modules/audit/repository.py | 14 + backend/app/modules/audit/schemas.py | 159 ++++++++ backend/app/modules/audit/service.py | 65 ++++ backend/app/modules/tasks/models.py | 31 +- backend/app/modules/tasks/repository.py | 14 +- backend/tests/test_alembic.py | 343 ++++++++++++++++++ backend/tests/test_audit.py | 271 ++++++++++++++ backend/tests/test_tasks.py | 50 ++- docs/architecture_data_model.md | 20 +- docs/operations_authorization_service.md | 31 ++ 17 files changed, 1241 insertions(+), 46 deletions(-) create mode 100644 backend/alembic/versions/0018_authority_audit_evidence.py create mode 100644 backend/app/modules/audit/schemas.py create mode 100644 backend/app/modules/audit/service.py create mode 100644 backend/tests/test_audit.py diff --git a/.agent-loop/LOOP_STATE.md b/.agent-loop/LOOP_STATE.md index 178615f9..d18ddf6d 100644 --- a/.agent-loop/LOOP_STATE.md +++ b/.agent-loop/LOOP_STATE.md @@ -3,23 +3,24 @@ ## Current State - Active initiative: `WS-AUTH-001` - Workstream Authorization Service -- Active planning chunk: `WS-AUTH-001-05A` - Shared Audit Ownership And +- Active planning chunk: none +- Active implementation chunk: `WS-AUTH-001-05A` - Shared Audit Ownership And Append-Only Authority Evidence -- Active implementation chunk: none - Branch: `codex/ws-auth-001-05-authority-evidence` - Worktree: `/home/abiorh/flow/workstream-auth-001-05` - Status: AUTH-04B post-merge memory merged through PR #114 as `97cd0f5`. - The user explicitly started AUTH-05. Required L1 plan review rejected the - combined contract before runtime edits and required the 05A/05B split. + The user explicitly started AUTH-05. Required L1 review rejected the combined + contract, then passed repaired AUTH-05A at `7a9023b`; bounded implementation + and deterministic evidence are active. - Prior `WS-AUTH-001-01` reviewed implementation SHA: `be0b836` - Prior `WS-AUTH-001-01` final merged branch head: `b5217e1` - Latest integrated `main` merge commit: `97cd0f5` -- Current gate: repair and re-review the AUTH-05A and AUTH-05B contracts before - any runtime edit. +- Current gate: AUTH-05A implementation evidence and exact-head internal review. - Size checkpoint: each child has a 350-line inspection and a 500-line production hard stop. -- Next chunk: AUTH-05A only after its repaired preimplementation review passes. - AUTH-05B and AUTH-06 remain inactive. +- Next chunk: AUTH-05B remains inactive until AUTH-05A merges, post-merge memory + is reconciled, and the user gives a separate explicit start. AUTH-06 remains + inactive. - Focused evidence: 77 isolated rate/config tests pass at 97 percent subsystem coverage, 59 final config/object-graph tests pass, and the exact migration proof and real API E2E pass. Final GitHub Backend passed 937 tests at 82.15 diff --git a/.agent-loop/REVIEW_LOG.md b/.agent-loop/REVIEW_LOG.md index e00bd432..397d4eed 100644 --- a/.agent-loop/REVIEW_LOG.md +++ b/.agent-loop/REVIEW_LOG.md @@ -1,5 +1,16 @@ # Review Log +## 2026-07-14 - WS-AUTH-001-05A Preimplementation Review Passed + +The repaired AUTH-05A contract passed required senior engineering, +QA/CI/test-delta, security/privacy, product/ops, architecture, docs, and reuse +review at `7a9023b`. The contract fixes migration ownership at `0018`, retains +the existing `audit_events` ledger and `AuditRepository` as sole persistence +owner, defines exact legacy/authority compatibility and bounded event shapes, +and requires unconditional normal-DML append-only triggers plus downgrade and +production-role custody proof. Bounded implementation and deterministic +evidence are active; AUTH-05B remains inactive. + ## 2026-07-14 - WS-AUTH-001-05 Plan Review Split Required After AUTH-04B post-merge memory merged through PR #114 as `97cd0f5`, the user diff --git a/.agent-loop/WORK_QUEUE.md b/.agent-loop/WORK_QUEUE.md index 6714066d..e003c03d 100644 --- a/.agent-loop/WORK_QUEUE.md +++ b/.agent-loop/WORK_QUEUE.md @@ -4,7 +4,7 @@ | Chunk | Title | Risk | Status | |---|---|---:|---| -| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | Contract repair and preimplementation re-review; no runtime implementation active | +| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | Repaired contract passed at `7a9023b`; bounded implementation and evidence active | ## Planned Next diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md index 288a4d3a..6ce3a9de 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md @@ -19,7 +19,7 @@ stopped. | `WS-AUTH-001-04A` | Request And Error Context | L1 | Merged through PR #111 as `90c9a28` | | `WS-AUTH-001-04B` | PostgreSQL Rate Controls | L1 | Merged through PR #113 as `05a63c8` | | `WS-AUTH-001-05` | Authority Evidence And Idempotency Foundation | L1 | Split before implementation into 05A and 05B | -| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | Contract repair and preimplementation re-review | +| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | Implementation and deterministic evidence | | `WS-AUTH-001-05B` | Authority Idempotency And Invalidation Foundation | L1 | Inactive pending 05A merge/memory and separate explicit start | | `WS-AUTH-001-06` | Canonical Actor Profile And Identity Link | L1 | Proposed | | `WS-AUTH-001-07` | Authorization Kernel And Permission Registry | L1 | Proposed | diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md index 12887c2e..7c7fbc6a 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md @@ -46,17 +46,16 @@ explicit human approval merged PR #113 to `main` as `05a63c8` on 2026-07-14. AUTH-04B post-merge memory merged through PR #114 as `97cd0f5`. The user then explicitly started AUTH-05. Required L1 plan review rejected the combined audit/idempotency contract before runtime edits and required children 05A and -05B. Contract repair and re-review are active; no runtime implementation is -active. +05B. The repaired AUTH-05A contract passed at `7a9023b`; its bounded runtime +implementation and deterministic evidence are active. ## Active planning chunk -`WS-AUTH-001-05A` - Shared Audit Ownership And Append-Only Authority Evidence. -Contract repair and required preimplementation re-review only. +None. ## Active implementation chunk -None. +`WS-AUTH-001-05A` - Shared Audit Ownership And Append-Only Authority Evidence. ## Current implementation branch @@ -75,7 +74,7 @@ None. | `WS-AUTH-001-04A` | Merged | `codex/ws-auth-001-04-request-api-controls` | #111 | Merged as `90c9a28`; production review `cdcaf77`; final branch head `36c4aa5`. | | `WS-AUTH-001-04B` | Merged | `codex/ws-auth-001-04b-postgres-rate-controls` | #113 | Merged as `05a63c8`; production review `67484b5`; final branch head `94fb2fe`. | | `WS-AUTH-001-05` | Split | `codex/ws-auth-001-05-authority-evidence` | - | Parent split before implementation into 05A and 05B. | -| `WS-AUTH-001-05A` | Contract review | `codex/ws-auth-001-05-authority-evidence` | - | Shared audit ownership and append-only authority evidence; runtime edits prohibited pending review. | +| `WS-AUTH-001-05A` | Implementation/evidence | `codex/ws-auth-001-05-authority-evidence` | - | Repaired contract passed at `7a9023b`; shared audit ownership and append-only authority evidence active. | | `WS-AUTH-001-05B` | Inactive | - | - | Idempotency and invalidation; requires 05A merge/memory and separate explicit start. | | `WS-AUTH-001-06` | Proposed | - | - | Canonical actor profile and identity link. | | `WS-AUTH-001-07` | Proposed | - | - | Authorization kernel and permissions. | @@ -91,11 +90,11 @@ None. ## Blockers -AUTH-05A runtime implementation is blocked on repaired-contract review. The -combined AUTH-05 contract was rejected before code changes because migration -`0017` is already owned by AUTH-04B and the shared-audit plus idempotency scope -was not reviewable as one L1 change. AUTH-05A now owns proposed migration -`0018`; AUTH-05B later owns proposed migration `0019`. Non-test +AUTH-05A has no preimplementation blocker. The combined AUTH-05 contract was +rejected before code changes because migration `0017` is already owned by +AUTH-04B and the shared-audit plus idempotency scope was not reviewable as one +L1 change. Repaired AUTH-05A owns migration `0018`; AUTH-05B later owns +migration `0019`. Non-test operators must later supply explicit classification evidence rather than inferred kinds before the owning canonical actor migration. diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md index ee08c226..78984f01 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md @@ -2,8 +2,8 @@ ## Status -Active for contract repair and required preimplementation review. Runtime edits -remain prohibited until that review passes. +Active implementation and deterministic evidence. Repaired preimplementation +review passed on contract head `7a9023b`. ## Parent initiative diff --git a/backend/alembic/versions/0018_authority_audit_evidence.py b/backend/alembic/versions/0018_authority_audit_evidence.py new file mode 100644 index 00000000..05d142d1 --- /dev/null +++ b/backend/alembic/versions/0018_authority_audit_evidence.py @@ -0,0 +1,232 @@ +"""add append-only authority audit evidence + +Revision ID: 0018_authority_audit_evidence +Revises: 0017_api_controls +Create Date: 2026-07-14 +""" + +from alembic import op +import sqlalchemy as sa + +revision = "0018_authority_audit_evidence" +down_revision = "0017_api_controls" +branch_labels = None +depends_on = None + +AUTHORITY_EVENT_TYPES = ( + "ActorProfileProvisioned", + "ServiceActorProvisioned", + "ActorIdentityLinked", + "ActorIdentityLinkRevoked", + "ActorIdentityLinkReactivated", + "ActorProfileSuspended", + "ActorProfileReactivated", + "ActorProfileDeactivated", + "InitialAccessAdministratorBootstrapped", + "AdminRoleGrantIssued", + "AdminRoleGrantRevoked", + "AdminRoleGrantIssueDenied", + "LastAccessAdministratorOperationDenied", + "ProjectRoleQualificationSnapshotCaptured", + "ProjectRoleGrantIssued", + "ProjectRoleGrantReplaced", + "ProjectRoleGrantRevoked", + "SensitiveAuthorizationAllowed", + "SensitiveAuthorizationDenied", + "AuthorityInvalidationRequested", +) + + +def upgrade() -> None: + """Extend the shared audit table and install normal-DML custody guards.""" + columns = ( + sa.Column( + "event_domain", + sa.String(24), + nullable=False, + server_default="legacy_lifecycle", + ), + sa.Column("event_version", sa.Integer(), nullable=True), + sa.Column("occurred_at", sa.DateTime(timezone=True), nullable=True), + sa.Column("actor_ref_kind", sa.String(32), nullable=True), + sa.Column("request_id", sa.Uuid(), nullable=True), + sa.Column("correlation_id", sa.Uuid(), nullable=True), + sa.Column("target_actor_ref_kind", sa.String(32), nullable=True), + sa.Column("target_actor_ref", sa.String(100), nullable=True), + sa.Column("matched_grant_id", sa.String(100), nullable=True), + sa.Column("permission_id", sa.String(120), nullable=True), + sa.Column("project_id", sa.String(36), nullable=True), + sa.Column("resource_type", sa.String(80), nullable=True), + sa.Column("resource_id", sa.String(100), nullable=True), + sa.Column("target_ref_kind", sa.String(32), nullable=True), + sa.Column("target_ref_id", sa.String(100), nullable=True), + sa.Column("denial_code", sa.String(80), nullable=True), + sa.Column("idempotency_reference", sa.Uuid(), nullable=True), + sa.Column("invalidation_cause_event_id", sa.String(36), nullable=True), + sa.Column("invalidation_target_kind", sa.String(32), nullable=True), + sa.Column("invalidation_target_ref", sa.String(100), nullable=True), + sa.Column("before_facts", sa.JSON(), nullable=True), + sa.Column("after_facts", sa.JSON(), nullable=True), + ) + for column in columns: + op.add_column("audit_events", column) + op.alter_column("audit_events", "external_subject", existing_type=sa.String(200), nullable=True) + op.alter_column("audit_events", "external_issuer", existing_type=sa.String(200), nullable=True) + + event_tokens = ", ".join(f"'{value}'" for value in AUTHORITY_EVENT_TYPES) + op.create_check_constraint( + "domain_shape", + "audit_events", + """ + (event_domain = 'legacy_lifecycle' and event_version is null and occurred_at is null + and actor_ref_kind is null and request_id is null and correlation_id is null + and target_actor_ref_kind is null and target_actor_ref is null + and matched_grant_id is null and permission_id is null and project_id is null + and resource_type is null and resource_id is null + and target_ref_kind is null and target_ref_id is null and denial_code is null + and idempotency_reference is null and invalidation_cause_event_id is null + and invalidation_target_kind is null and invalidation_target_ref is null + and before_facts is null and after_facts is null + and external_subject is not null and external_issuer is not null) + or + (event_domain = 'authority' and event_version = 1 and occurred_at is not null + and actor_ref_kind in ('legacy_actor', 'actor_profile', 'system_principal') + and request_id is not null and correlation_id is not null + and from_status is null and to_status is null + and (reason is null or length(reason) <= 1000) + and external_subject is null and external_issuer is null + and actor_roles::jsonb = '[]'::jsonb and claim_snapshot::jsonb = '{}'::jsonb + and auth_source = 'local_authority' and is_dev_auth = false + and event_payload::jsonb = '{}'::jsonb) + """, + ) + op.create_check_constraint( + "authority_tokens", + "audit_events", + f"event_domain <> 'authority' or event_type in ({event_tokens})", + ) + op.create_check_constraint( + "reference_pairs", + "audit_events", + """ + (target_actor_ref_kind is null) = (target_actor_ref is null) + and (resource_type is null) = (resource_id is null) + and (target_ref_kind is null) = (target_ref_id is null) + and (invalidation_target_kind is null) = (invalidation_target_ref is null) + """, + ) + op.create_check_constraint( + "fact_bounds", + "audit_events", + """ + (before_facts is null or + (json_typeof(before_facts) = 'object' and octet_length(before_facts::text) <= 4096)) + and (after_facts is null or + (json_typeof(after_facts) = 'object' and octet_length(after_facts::text) <= 4096)) + """, + ) + op.create_check_constraint( + "foundation_shapes", + "audit_events", + """ + event_domain <> 'authority' + or event_type not in ('SensitiveAuthorizationAllowed', + 'SensitiveAuthorizationDenied', 'AuthorityInvalidationRequested') + or (event_type = 'SensitiveAuthorizationAllowed' and permission_id is not null + and denial_code is null and invalidation_cause_event_id is null + and invalidation_target_kind is null) + or (event_type = 'SensitiveAuthorizationDenied' and permission_id is not null + and denial_code is not null and invalidation_cause_event_id is null + and invalidation_target_kind is null and idempotency_reference is null) + or (event_type = 'AuthorityInvalidationRequested' + and invalidation_cause_event_id is not null + and invalidation_target_kind is not null and denial_code is null) + """, + ) + for name, fields in ( + ("ix_audit_events_request_id", ["request_id"]), + ("ix_audit_events_correlation_id", ["correlation_id"]), + ("ix_audit_events_occurred_at", ["occurred_at"]), + ("ix_audit_events_project_id", ["project_id"]), + ("ix_audit_events_actor_ref", ["actor_ref_kind", "actor_id"]), + ): + op.create_index(name, "audit_events", fields) + + op.execute(""" + create function set_authority_audit_database_time() returns trigger language plpgsql as $$ + begin + if new.event_domain = 'authority' then + new.occurred_at = statement_timestamp(); + else + new.occurred_at = null; + end if; + return new; + end $$ + """) + op.execute(""" + create trigger audit_events_set_authority_time before insert on audit_events + for each row execute function set_authority_audit_database_time() + """) + op.execute(""" + create function reject_audit_event_mutation() returns trigger language plpgsql as $$ + begin + raise exception 'audit events are append-only' using errcode = '55000'; + end $$ + """) + op.execute(""" + create trigger audit_events_reject_update_delete before update or delete on audit_events + for each row execute function reject_audit_event_mutation() + """) + op.execute(""" + create trigger audit_events_reject_truncate before truncate on audit_events + for each statement execute function reject_audit_event_mutation() + """) + + +def downgrade() -> None: + """Remove only unused authority-envelope schema while preserving legacy rows.""" + bind = op.get_bind() + bind.execute(sa.text("lock table audit_events in access exclusive mode")) + has_authority = bind.execute( + sa.text("select exists(select 1 from audit_events where event_domain = 'authority')") + ).scalar_one() + if has_authority: + raise RuntimeError("cannot downgrade non-empty authority audit evidence") + + for trigger in ( + "audit_events_reject_truncate", + "audit_events_reject_update_delete", + "audit_events_set_authority_time", + ): + op.execute(f"drop trigger {trigger} on audit_events") + op.execute("drop function reject_audit_event_mutation()") + op.execute("drop function set_authority_audit_database_time()") + for name in ( + "ix_audit_events_actor_ref", + "ix_audit_events_project_id", + "ix_audit_events_occurred_at", + "ix_audit_events_correlation_id", + "ix_audit_events_request_id", + ): + op.drop_index(name, table_name="audit_events") + for name in ( + "foundation_shapes", + "fact_bounds", + "reference_pairs", + "authority_tokens", + "domain_shape", + ): + op.drop_constraint(name, "audit_events", type_="check") + for column in reversed( + ( + "event_version", "occurred_at", "actor_ref_kind", "request_id", "correlation_id", + "target_actor_ref_kind", "target_actor_ref", "matched_grant_id", "permission_id", + "project_id", "resource_type", "resource_id", "target_ref_kind", "target_ref_id", + "denial_code", "idempotency_reference", "invalidation_cause_event_id", + "invalidation_target_kind", "invalidation_target_ref", "before_facts", "after_facts", + "event_domain", + ) + ): + op.drop_column("audit_events", column) + op.alter_column("audit_events", "external_issuer", existing_type=sa.String(200), nullable=False) + op.alter_column("audit_events", "external_subject", existing_type=sa.String(200), nullable=False) diff --git a/backend/app/modules/audit/repository.py b/backend/app/modules/audit/repository.py index a1d2e75c..25b97688 100644 --- a/backend/app/modules/audit/repository.py +++ b/backend/app/modules/audit/repository.py @@ -2,6 +2,9 @@ from __future__ import annotations +from collections.abc import Sequence + +from sqlalchemy import select from sqlalchemy.ext.asyncio import AsyncSession from app.modules.tasks.models import AuditEvent @@ -31,3 +34,14 @@ async def add_audit_event(self, event: AuditEvent) -> AuditEvent: await self._session.flush() await self._session.refresh(event) return event + + async def list_audit_events( + self, entity_type: str, entity_id: str + ) -> Sequence[AuditEvent]: + """List events for one entity in database creation order.""" + result = await self._session.execute( + select(AuditEvent) + .where(AuditEvent.entity_type == entity_type, AuditEvent.entity_id == entity_id) + .order_by(AuditEvent.created_at.asc(), AuditEvent.id.asc()) + ) + return result.scalars().all() diff --git a/backend/app/modules/audit/schemas.py b/backend/app/modules/audit/schemas.py new file mode 100644 index 00000000..dda5441f --- /dev/null +++ b/backend/app/modules/audit/schemas.py @@ -0,0 +1,159 @@ +"""Typed, privacy-bounded authority audit inputs.""" + +from __future__ import annotations + +from enum import StrEnum +import json +import re +from typing import Self +from uuid import UUID, uuid4 + +from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator + +_FACT_KEY = re.compile(r"^[a-z][a-z0-9_]{0,63}$") + + +class AuthorityEventType(StrEnum): + """Closed authority event tokens from the adopted specification.""" + + ACTOR_PROFILE_PROVISIONED = "ActorProfileProvisioned" + SERVICE_ACTOR_PROVISIONED = "ServiceActorProvisioned" + ACTOR_IDENTITY_LINKED = "ActorIdentityLinked" + ACTOR_IDENTITY_LINK_REVOKED = "ActorIdentityLinkRevoked" + ACTOR_IDENTITY_LINK_REACTIVATED = "ActorIdentityLinkReactivated" + ACTOR_PROFILE_SUSPENDED = "ActorProfileSuspended" + ACTOR_PROFILE_REACTIVATED = "ActorProfileReactivated" + ACTOR_PROFILE_DEACTIVATED = "ActorProfileDeactivated" + INITIAL_ACCESS_ADMIN_BOOTSTRAPPED = "InitialAccessAdministratorBootstrapped" + ADMIN_ROLE_GRANT_ISSUED = "AdminRoleGrantIssued" + ADMIN_ROLE_GRANT_REVOKED = "AdminRoleGrantRevoked" + ADMIN_ROLE_GRANT_ISSUE_DENIED = "AdminRoleGrantIssueDenied" + LAST_ACCESS_ADMIN_OPERATION_DENIED = "LastAccessAdministratorOperationDenied" + PROJECT_ROLE_QUALIFICATION_CAPTURED = "ProjectRoleQualificationSnapshotCaptured" + PROJECT_ROLE_GRANT_ISSUED = "ProjectRoleGrantIssued" + PROJECT_ROLE_GRANT_REPLACED = "ProjectRoleGrantReplaced" + PROJECT_ROLE_GRANT_REVOKED = "ProjectRoleGrantRevoked" + SENSITIVE_AUTHORIZATION_ALLOWED = "SensitiveAuthorizationAllowed" + SENSITIVE_AUTHORIZATION_DENIED = "SensitiveAuthorizationDenied" + AUTHORITY_INVALIDATION_REQUESTED = "AuthorityInvalidationRequested" + + +# Specification categories are contiguous in the closed enum above. +_EVENT_TYPES = tuple(AuthorityEventType) +_EVENT_FACT_KEYS = { + **dict.fromkeys( + _EVENT_TYPES[:8], + frozenset({"status", "subject_kind", "provisioning_method", "link_status"}), + ), + **dict.fromkeys( + _EVENT_TYPES[8:17], + frozenset( + {"status", "role_id", "scope_type", "scope_id", "effective", "qualification_status"} + ), + ), + **dict.fromkeys( + _EVENT_TYPES[17:19], frozenset({"status", "decision_code", "effective"}) + ), + _EVENT_TYPES[19]: frozenset({"status", "target_status", "effective"}), +} + + +class ActorReferenceKind(StrEnum): + """Stable namespaces usable before and after canonical actor migration.""" + + LEGACY_ACTOR = "legacy_actor" + ACTOR_PROFILE = "actor_profile" + SYSTEM_PRINCIPAL = "system_principal" + + +class AuthorityAuditEventInput(BaseModel): + """Validate one authority event without provider claims or request bodies.""" + + model_config = ConfigDict(extra="forbid", strict=True) + + event_id: UUID = Field(default_factory=uuid4) + event_type: AuthorityEventType + entity_type: str = Field(min_length=1, max_length=80, pattern=r"^[a-z][a-z0-9_.:-]*$") + entity_id: str = Field(min_length=1, max_length=36) + actor_ref_kind: ActorReferenceKind + actor_ref: str = Field(min_length=1, max_length=100) + request_id: UUID + correlation_id: UUID + target_actor_ref_kind: ActorReferenceKind | None = None + target_actor_ref: str | None = Field(default=None, min_length=1, max_length=100) + matched_grant_id: str | None = Field(default=None, min_length=1, max_length=100) + permission_id: str | None = Field( + default=None, min_length=1, max_length=120, pattern=r"^[a-z][a-z0-9_.:-]*$" + ) + project_id: str | None = Field(default=None, min_length=1, max_length=36) + resource_type: str | None = Field( + default=None, min_length=1, max_length=80, pattern=r"^[a-z][a-z0-9_.:-]*$" + ) + resource_id: str | None = Field(default=None, min_length=1, max_length=100) + target_ref_kind: str | None = Field( + default=None, min_length=1, max_length=32, pattern=r"^[a-z][a-z0-9_.:-]*$" + ) + target_ref_id: str | None = Field(default=None, min_length=1, max_length=100) + reason: str | None = Field(default=None, max_length=1000) + denial_code: str | None = Field( + default=None, min_length=1, max_length=80, pattern=r"^[a-z][a-z0-9_]*$" + ) + idempotency_reference: UUID | None = None + invalidation_cause_event_id: UUID | None = None + invalidation_target_kind: str | None = Field( + default=None, min_length=1, max_length=32, pattern=r"^[a-z][a-z0-9_.:-]*$" + ) + invalidation_target_ref: str | None = Field(default=None, min_length=1, max_length=100) + before_facts: dict[str, str | int | bool | None] | None = None + after_facts: dict[str, str | int | bool | None] | None = None + + @field_validator("before_facts", "after_facts", mode="before") + @classmethod + def validate_facts(cls, value: object) -> object: + """Reject nested, excessive, or oversized state facts.""" + if value is None: + return None + if not isinstance(value, dict) or any( + not isinstance(key, str) for key in value + ): + raise ValueError("invalid authority state facts") + if len(value) > 16 or any(_FACT_KEY.fullmatch(key) is None for key in value): + raise ValueError("invalid authority state facts") + if any(isinstance(item, (dict, list, tuple, set)) for item in value.values()): + raise ValueError("invalid authority state facts") + if len(json.dumps(value, separators=(",", ":"), ensure_ascii=False).encode()) > 4096: + raise ValueError("authority state facts exceed size limit") + return value + + @model_validator(mode="after") + def validate_shape(self) -> Self: + """Enforce paired references and foundation event-specific fields.""" + for left, right in ( + (self.target_actor_ref_kind, self.target_actor_ref), + (self.resource_type, self.resource_id), + (self.target_ref_kind, self.target_ref_id), + (self.invalidation_target_kind, self.invalidation_target_ref), + ): + if (left is None) != (right is None): + raise ValueError("authority reference fields must be paired") + invalidation = self.invalidation_cause_event_id is not None or self.invalidation_target_kind + allowed_fact_keys = _EVENT_FACT_KEYS[self.event_type] + if any( + facts is not None and not set(facts).issubset(allowed_fact_keys) + for facts in (self.before_facts, self.after_facts) + ): + raise ValueError("authority state facts are not allowed for this event") + if self.event_type == AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED: + if self.permission_id is None or self.denial_code is not None or invalidation: + raise ValueError("invalid allowed authorization evidence") + elif self.event_type == AuthorityEventType.SENSITIVE_AUTHORIZATION_DENIED: + if self.permission_id is None or self.denial_code is None or invalidation: + raise ValueError("invalid denied authorization evidence") + if self.idempotency_reference is not None: + raise ValueError("denial cannot reference committed idempotency") + elif self.event_type == AuthorityEventType.AUTHORITY_INVALIDATION_REQUESTED: + if self.invalidation_cause_event_id is None or self.invalidation_target_kind is None: + raise ValueError("invalid authority invalidation evidence") + if self.denial_code is not None: + raise ValueError("invalidation cannot carry denial code") + return self diff --git a/backend/app/modules/audit/service.py b/backend/app/modules/audit/service.py new file mode 100644 index 00000000..6f2945f2 --- /dev/null +++ b/backend/app/modules/audit/service.py @@ -0,0 +1,65 @@ +"""Application service for typed authority audit evidence.""" + +from sqlalchemy.ext.asyncio import AsyncSession + +from app.modules.audit.repository import AuditRepository +from app.modules.audit.schemas import AuthorityAuditEventInput +from app.modules.tasks.models import AuditEvent + + +class AuditService: + """Build and persist privacy-safe authority events in the caller unit of work.""" + + def __init__(self, session: AsyncSession) -> None: + """Bind the shared audit repository to the caller's session.""" + self._repository = AuditRepository(session) + + async def add_authority_event(self, value: AuthorityAuditEventInput) -> AuditEvent: + """Persist one validated authority event without committing its transaction.""" + event = AuditEvent( + id=str(value.event_id), + entity_type=value.entity_type, + entity_id=value.entity_id, + event_type=value.event_type.value, + from_status=None, + to_status=None, + actor_id=value.actor_ref, + external_subject=None, + external_issuer=None, + actor_roles=[], + claim_snapshot={}, + auth_source="local_authority", + is_dev_auth=False, + reason=value.reason, + event_payload={}, + event_domain="authority", + event_version=1, + actor_ref_kind=value.actor_ref_kind.value, + request_id=str(value.request_id), + correlation_id=str(value.correlation_id), + target_actor_ref_kind=( + value.target_actor_ref_kind.value if value.target_actor_ref_kind else None + ), + target_actor_ref=value.target_actor_ref, + matched_grant_id=value.matched_grant_id, + permission_id=value.permission_id, + project_id=value.project_id, + resource_type=value.resource_type, + resource_id=value.resource_id, + target_ref_kind=value.target_ref_kind, + target_ref_id=value.target_ref_id, + denial_code=value.denial_code, + idempotency_reference=( + str(value.idempotency_reference) if value.idempotency_reference else None + ), + invalidation_cause_event_id=( + str(value.invalidation_cause_event_id) + if value.invalidation_cause_event_id + else None + ), + invalidation_target_kind=value.invalidation_target_kind, + invalidation_target_ref=value.invalidation_target_ref, + before_facts=value.before_facts, + after_facts=value.after_facts, + ) + return await self._repository.add_audit_event(event) diff --git a/backend/app/modules/tasks/models.py b/backend/app/modules/tasks/models.py index 19d88d61..64f13704 100644 --- a/backend/app/modules/tasks/models.py +++ b/backend/app/modules/tasks/models.py @@ -18,6 +18,7 @@ String, Text, UniqueConstraint, + Uuid, text, ) from sqlalchemy.orm import Mapped, mapped_column, relationship @@ -453,7 +454,7 @@ class EvidenceItem(Base): class AuditEvent(Base): - """Audit event for actor-attributed task lifecycle changes.""" + """Shared append-only lifecycle and authority audit evidence.""" __tablename__ = "audit_events" @@ -464,8 +465,8 @@ class AuditEvent(Base): from_status: Mapped[str | None] = mapped_column(String(30)) to_status: Mapped[str | None] = mapped_column(String(30)) actor_id: Mapped[str] = mapped_column(String(100), nullable=False, index=True) - external_subject: Mapped[str] = mapped_column(String(200), nullable=False) - external_issuer: Mapped[str] = mapped_column(String(200), nullable=False) + external_subject: Mapped[str | None] = mapped_column(String(200)) + external_issuer: Mapped[str | None] = mapped_column(String(200)) actor_roles: Mapped[list[str]] = mapped_column(JSON, nullable=False, default=list) claim_snapshot: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict) auth_source: Mapped[str] = mapped_column(String(30), nullable=False) @@ -473,3 +474,27 @@ class AuditEvent(Base): reason: Mapped[str | None] = mapped_column(Text) event_payload: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict) created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now()) + event_domain: Mapped[str] = mapped_column( + String(24), nullable=False, default="legacy_lifecycle", server_default="legacy_lifecycle" + ) + event_version: Mapped[int | None] = mapped_column(Integer) + occurred_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True)) + actor_ref_kind: Mapped[str | None] = mapped_column(String(32)) + request_id: Mapped[str | None] = mapped_column(Uuid(as_uuid=False)) + correlation_id: Mapped[str | None] = mapped_column(Uuid(as_uuid=False)) + target_actor_ref_kind: Mapped[str | None] = mapped_column(String(32)) + target_actor_ref: Mapped[str | None] = mapped_column(String(100)) + matched_grant_id: Mapped[str | None] = mapped_column(String(100)) + permission_id: Mapped[str | None] = mapped_column(String(120)) + project_id: Mapped[str | None] = mapped_column(String(36)) + resource_type: Mapped[str | None] = mapped_column(String(80)) + resource_id: Mapped[str | None] = mapped_column(String(100)) + target_ref_kind: Mapped[str | None] = mapped_column(String(32)) + target_ref_id: Mapped[str | None] = mapped_column(String(100)) + denial_code: Mapped[str | None] = mapped_column(String(80)) + idempotency_reference: Mapped[str | None] = mapped_column(Uuid(as_uuid=False)) + invalidation_cause_event_id: Mapped[str | None] = mapped_column(String(36)) + invalidation_target_kind: Mapped[str | None] = mapped_column(String(32)) + invalidation_target_ref: Mapped[str | None] = mapped_column(String(100)) + before_facts: Mapped[dict | None] = mapped_column(JSON(none_as_null=True)) + after_facts: Mapped[dict | None] = mapped_column(JSON(none_as_null=True)) diff --git a/backend/app/modules/tasks/repository.py b/backend/app/modules/tasks/repository.py index 983e2e88..57bcbd54 100644 --- a/backend/app/modules/tasks/repository.py +++ b/backend/app/modules/tasks/repository.py @@ -9,6 +9,7 @@ from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.orm import selectinload +from app.modules.audit.repository import AuditRepository from app.modules.tasks.models import ( AuditEvent, EvidenceItem, @@ -28,6 +29,7 @@ def __init__(self, session: AsyncSession) -> None: session: Async SQLAlchemy session for the current unit of work. """ self._session = session + self._audit_repository = AuditRepository(session) async def add_task(self, task: WorkstreamTask) -> WorkstreamTask: """Persist a new task and refresh generated database fields. @@ -207,10 +209,7 @@ async def add_audit_event(self, event: AuditEvent) -> AuditEvent: Returns: Persisted audit event model. """ - self._session.add(event) - await self._session.flush() - await self._session.refresh(event) - return event + return await self._audit_repository.add_audit_event(event) async def list_audit_events(self, entity_type: str, entity_id: str) -> Sequence[AuditEvent]: """List audit events for one entity in creation order. @@ -222,9 +221,4 @@ async def list_audit_events(self, entity_type: str, entity_id: str) -> Sequence[ Returns: Matching audit events ordered by creation time. """ - result = await self._session.execute( - select(AuditEvent) - .where(AuditEvent.entity_type == entity_type, AuditEvent.entity_id == entity_id) - .order_by(AuditEvent.created_at.asc(), AuditEvent.id.asc()) - ) - return result.scalars().all() + return await self._audit_repository.list_audit_events(entity_type, entity_id) diff --git a/backend/tests/test_alembic.py b/backend/tests/test_alembic.py index 646b35f7..5d9d6b9a 100644 --- a/backend/tests/test_alembic.py +++ b/backend/tests/test_alembic.py @@ -410,6 +410,119 @@ def guarded_downgrade() -> None: } +def test_authority_audit_schema_preserves_legacy_and_guards_downgrade( + isolated_database_env: str, + migration_lock, +) -> None: + """Prove 0018 preserves legacy evidence and refuses destructive downgrade.""" + project_root = Path(__file__).resolve().parents[1] + config = Config(str(project_root / "alembic.ini")) + config.set_main_option("script_location", str(project_root / "alembic")) + legacy_id = str(uuid4()) + authority_id = str(uuid4()) + + with migration_lock(): + try: + command.downgrade(config, "base") + command.upgrade(config, "0017_api_controls") + before = asyncio.run( + _seed_and_fetch_legacy_audit(isolated_database_env, legacy_id) + ) + + command.upgrade(config, "0018_authority_audit_evidence") + after = asyncio.run(_fetch_audit_row(isolated_database_env, legacy_id)) + schema = asyncio.run(_authority_audit_schema(isolated_database_env)) + occurred_at = asyncio.run( + _insert_authority_audit_fixture( + isolated_database_env, authority_id + ) + ) + + with pytest.raises(RuntimeError, match="non-empty authority audit"): + command.downgrade(config, "0017_api_controls") + refused = asyncio.run(_authority_audit_state(isolated_database_env)) + + asyncio.run( + _remove_authority_audit_fixture( + isolated_database_env, authority_id + ) + ) + command.downgrade(config, "0017_api_controls") + downgraded = asyncio.run(_fetch_audit_row(isolated_database_env, legacy_id)) + command.upgrade(config, "0018_authority_audit_evidence") + restored_schema = asyncio.run( + _authority_audit_schema(isolated_database_env) + ) + finally: + asyncio.run( + _remove_authority_audit_fixture( + isolated_database_env, authority_id + ) + ) + command.downgrade(config, "base") + command.upgrade(config, "head") + + assert after == {**before, "event_domain": "legacy_lifecycle"} + assert downgraded == before + assert occurred_at.year >= 2026 + assert refused == { + "revision": "0018_authority_audit_evidence", + "authority_rows": 1, + } + assert schema == restored_schema + assert schema == { + "columns": { + "actor_ref_kind:varchar:YES", + "after_facts:json:YES", + "before_facts:json:YES", + "correlation_id:uuid:YES", + "denial_code:varchar:YES", + "event_domain:varchar:NO", + "event_version:int4:YES", + "idempotency_reference:uuid:YES", + "invalidation_cause_event_id:varchar:YES", + "invalidation_target_kind:varchar:YES", + "invalidation_target_ref:varchar:YES", + "matched_grant_id:varchar:YES", + "occurred_at:timestamptz:YES", + "permission_id:varchar:YES", + "project_id:varchar:YES", + "request_id:uuid:YES", + "resource_id:varchar:YES", + "resource_type:varchar:YES", + "target_actor_ref:varchar:YES", + "target_actor_ref_kind:varchar:YES", + "target_ref_id:varchar:YES", + "target_ref_kind:varchar:YES", + }, + "constraints": { + "ck_audit_events_authority_tokens", + "ck_audit_events_domain_shape", + "ck_audit_events_fact_bounds", + "ck_audit_events_foundation_shapes", + "ck_audit_events_reference_pairs", + }, + "indexes": { + "ix_audit_events_actor_ref", + "ix_audit_events_correlation_id", + "ix_audit_events_occurred_at", + "ix_audit_events_project_id", + "ix_audit_events_request_id", + }, + "triggers": { + "audit_events_reject_truncate", + "audit_events_reject_update_delete", + "audit_events_set_authority_time", + }, + "functions": { + "reject_audit_event_mutation", + "set_authority_audit_database_time", + }, + "legacy_default": True, + "external_identity_nullable": True, + } + + async def _fetch_columns(database_url: str) -> set[str]: """Return current public table columns as table.column names.""" engine = create_async_engine(database_url) @@ -2006,3 +2119,233 @@ async def _clear_api_rate_controls(database_url: str) -> None: await connection.execute(text("delete from api_rate_control_counters")) finally: await engine.dispose() + + +async def _seed_and_fetch_legacy_audit( + database_url: str, event_id: str +) -> dict[str, object]: + """Insert one prior-head lifecycle event and return its legacy fields.""" + engine = create_async_engine(database_url) + try: + async with engine.begin() as connection: + await connection.execute( + text( + "insert into audit_events " + "(id, entity_type, entity_id, event_type, from_status, to_status, " + "actor_id, external_subject, external_issuer, actor_roles, " + "claim_snapshot, auth_source, is_dev_auth, reason, event_payload) " + "values (:id, 'task', :entity_id, 'task_created', null, 'draft', " + "'legacy-actor', 'opaque-subject', 'https://issuer.example.test', " + "'[\"project_manager\"]'::json, '{\"bounded\": true}'::json, " + "'verified_token', false, 'created', '{\"source\": \"manual\"}'::json)" + ), + {"id": event_id, "entity_id": str(uuid4())}, + ) + finally: + await engine.dispose() + return await _fetch_audit_row(database_url, event_id) + + +async def _fetch_audit_row(database_url: str, event_id: str) -> dict[str, object]: + """Fetch stable legacy fields and the authority domain when it exists.""" + engine = create_async_engine(database_url) + try: + async with engine.begin() as connection: + has_domain = await connection.scalar( + text( + "select exists(select 1 from information_schema.columns " + "where table_name = 'audit_events' and column_name = 'event_domain')" + ) + ) + domain = ", event_domain" if has_domain else "" + row = ( + await connection.execute( + text( + "select id, entity_type, entity_id, event_type, from_status, " + "to_status, actor_id, external_subject, external_issuer, " + "actor_roles, claim_snapshot, auth_source, is_dev_auth, reason, " + f"event_payload{domain} from audit_events where id = :id" + ), + {"id": event_id}, + ) + ).mappings().one() + return dict(row) + finally: + await engine.dispose() + + +async def _authority_audit_schema(database_url: str) -> dict[str, object]: + """Return the exact 0018 authority-audit schema surface.""" + new_columns = { + "event_domain", "event_version", "occurred_at", "actor_ref_kind", + "request_id", "correlation_id", "target_actor_ref_kind", "target_actor_ref", + "matched_grant_id", "permission_id", "project_id", "resource_type", + "resource_id", "target_ref_kind", "target_ref_id", "denial_code", + "idempotency_reference", "invalidation_cause_event_id", + "invalidation_target_kind", "invalidation_target_ref", "before_facts", + "after_facts", + } + engine = create_async_engine(database_url) + try: + async with engine.begin() as connection: + columns = ( + await connection.execute( + text( + "select column_name, udt_name, is_nullable, column_default " + "from information_schema.columns where table_schema = 'public' " + "and table_name = 'audit_events'" + ) + ) + ).mappings().all() + by_name = {row["column_name"]: row for row in columns} + constraints = set( + ( + await connection.execute( + text( + "select conname from pg_constraint where " + "conrelid = 'audit_events'::regclass " + "and conname like 'ck_audit_events_%'" + ) + ) + ).scalars() + ) + indexes = set( + ( + await connection.execute( + text( + "select indexname from pg_indexes where schemaname = 'public' " + "and tablename = 'audit_events' and indexname in " + "('ix_audit_events_request_id', 'ix_audit_events_correlation_id', " + "'ix_audit_events_occurred_at', 'ix_audit_events_project_id', " + "'ix_audit_events_actor_ref')" + ) + ) + ).scalars() + ) + triggers = set( + ( + await connection.execute( + text( + "select tgname from pg_trigger where " + "tgrelid = 'audit_events'::regclass and not tgisinternal" + ) + ) + ).scalars() + ) + functions = set( + ( + await connection.execute( + text( + "select proname from pg_proc where proname in " + "('reject_audit_event_mutation', " + "'set_authority_audit_database_time')" + ) + ) + ).scalars() + ) + return { + "columns": { + f"{name}:{by_name[name]['udt_name']}:{by_name[name]['is_nullable']}" + for name in new_columns + }, + "constraints": constraints, + "indexes": indexes, + "triggers": triggers, + "functions": functions, + "legacy_default": "legacy_lifecycle" + in (by_name["event_domain"]["column_default"] or ""), + "external_identity_nullable": all( + by_name[name]["is_nullable"] == "YES" + for name in ("external_subject", "external_issuer") + ), + } + finally: + await engine.dispose() + + +async def _insert_authority_audit_fixture( + database_url: str, event_id: str +): + """Insert valid authority evidence while proving database-owned time.""" + engine = create_async_engine(database_url) + try: + async with engine.begin() as connection: + return await connection.scalar( + text( + "insert into audit_events " + "(id, entity_type, entity_id, event_type, actor_id, actor_roles, " + "claim_snapshot, auth_source, is_dev_auth, event_payload, event_domain, " + "event_version, occurred_at, actor_ref_kind, request_id, correlation_id, " + "permission_id) values (:id, 'actor', :entity_id, " + "'SensitiveAuthorizationAllowed', 'workstream:system:test', '[]'::json, " + "'{}'::json, 'local_authority', false, '{}'::json, 'authority', 1, " + "'2000-01-01T00:00:00Z', 'system_principal', :request_id, " + ":correlation_id, 'actor.manage') returning occurred_at" + ), + { + "id": event_id, + "entity_id": str(uuid4()), + "request_id": str(uuid4()), + "correlation_id": str(uuid4()), + }, + ) + finally: + await engine.dispose() + + +async def _authority_audit_state(database_url: str) -> dict[str, object]: + """Return migration revision and retained authority evidence count.""" + engine = create_async_engine(database_url) + try: + async with engine.begin() as connection: + return { + "revision": await connection.scalar( + text("select version_num from alembic_version") + ), + "authority_rows": await connection.scalar( + text( + "select count(*) from audit_events " + "where event_domain = 'authority'" + ) + ), + } + finally: + await engine.dispose() + + +async def _remove_authority_audit_fixture( + database_url: str, event_id: str +) -> None: + """Perform explicit owner-only fixture cleanup under the documented lock.""" + engine = create_async_engine(database_url) + try: + async with engine.begin() as connection: + has_domain = await connection.scalar( + text( + "select exists(select 1 from information_schema.columns " + "where table_name = 'audit_events' and column_name = 'event_domain')" + ) + ) + if not has_domain: + return + await connection.execute(text("lock table audit_events in access exclusive mode")) + await connection.execute( + text( + "alter table audit_events disable trigger " + "audit_events_reject_update_delete" + ) + ) + await connection.execute( + text( + "delete from audit_events where id = :id and event_domain = 'authority'" + ), + {"id": event_id}, + ) + await connection.execute( + text( + "alter table audit_events enable trigger " + "audit_events_reject_update_delete" + ) + ) + finally: + await engine.dispose() diff --git a/backend/tests/test_audit.py b/backend/tests/test_audit.py new file mode 100644 index 00000000..d07ea984 --- /dev/null +++ b/backend/tests/test_audit.py @@ -0,0 +1,271 @@ +from __future__ import annotations + +from pathlib import Path +from uuid import UUID, uuid4 + +from alembic import command +from alembic.config import Config +from pydantic import ValidationError +import pytest +from sqlalchemy import text +from sqlalchemy.exc import DBAPIError, IntegrityError +from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine + +from app.modules.audit.repository import AuditRepository +from app.modules.audit.schemas import ( + ActorReferenceKind, + AuthorityAuditEventInput, + AuthorityEventType, +) +from app.modules.audit.service import AuditService +from app.modules.tasks.models import AuditEvent + + +@pytest.fixture +def audit_database_env(postgres_database_url: str, migration_lock) -> str: + """Ensure audit tests do not inherit another migration test's schema state.""" + project_root = Path(__file__).resolve().parents[1] + config = Config(str(project_root / "alembic.ini")) + config.set_main_option("script_location", str(project_root / "alembic")) + with migration_lock(): + command.upgrade(config, "head") + return postgres_database_url + + +@pytest.fixture +async def audit_factory(audit_database_env: str): + """Provide independent sessions over the isolated migrated database.""" + engine = create_async_engine(audit_database_env) + try: + yield async_sessionmaker(engine, expire_on_commit=False) + finally: + async with engine.begin() as connection: + await connection.execute( + text("lock table audit_events in access exclusive mode") + ) + await connection.execute( + text( + "alter table audit_events disable trigger " + "audit_events_reject_update_delete" + ) + ) + await connection.execute( + text("delete from audit_events where event_domain = 'authority'") + ) + await connection.execute( + text( + "alter table audit_events enable trigger " + "audit_events_reject_update_delete" + ) + ) + await engine.dispose() + + +def _authority_input(event_type: AuthorityEventType, **overrides) -> AuthorityAuditEventInput: + values = { + "event_type": event_type, + "entity_type": "actor", + "entity_id": str(uuid4()), + "actor_ref_kind": ActorReferenceKind.SYSTEM_PRINCIPAL, + "actor_ref": "workstream:system:test", + "request_id": uuid4(), + "correlation_id": uuid4(), + "permission_id": "actor.manage", + } + values.update(overrides) + return AuthorityAuditEventInput(**values) + + +def test_authority_input_rejects_unbounded_or_inconsistent_evidence() -> None: + with pytest.raises(ValidationError, match="Extra inputs are not permitted"): + _authority_input( + AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED, + raw_token="must-never-enter-evidence", + ) + with pytest.raises(ValidationError, match="reference fields must be paired"): + _authority_input( + AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED, + resource_type="project", + ) + with pytest.raises(ValidationError, match="state facts"): + _authority_input( + AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED, + before_facts={"nested": {"unsafe": True}}, + ) + with pytest.raises(ValidationError, match="exceed size limit"): + _authority_input( + AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED, + before_facts={"status": "x" * 4096}, + ) + with pytest.raises(ValidationError, match="not allowed for this event"): + _authority_input( + AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED, + before_facts={"policy_body": "must-never-enter-evidence"}, + ) + with pytest.raises(ValidationError, match="denial cannot reference"): + _authority_input( + AuthorityEventType.SENSITIVE_AUTHORIZATION_DENIED, + denial_code="actor_suspended", + idempotency_reference=uuid4(), + ) + with pytest.raises(ValidationError, match="invalidation evidence"): + _authority_input(AuthorityEventType.AUTHORITY_INVALIDATION_REQUESTED) + + +async def test_authority_writer_persists_typed_privacy_neutral_events(audit_factory) -> None: + future_idempotency = uuid4() + allowed = _authority_input( + AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED, + project_id=str(uuid4()), + resource_type="project", + resource_id=str(uuid4()), + before_facts={"status": "active"}, + ) + denied = _authority_input( + AuthorityEventType.SENSITIVE_AUTHORIZATION_DENIED, + denial_code="permission_denied", + reason="Local permission was not effective.", + ) + invalidation = _authority_input( + AuthorityEventType.AUTHORITY_INVALIDATION_REQUESTED, + permission_id=None, + invalidation_cause_event_id=allowed.event_id, + invalidation_target_kind="actor", + invalidation_target_ref="legacy:target", + idempotency_reference=future_idempotency, + after_facts={"status": "revoked"}, + ) + + async with audit_factory() as session: + service = AuditService(session) + stored = [ + await service.add_authority_event(value) + for value in (allowed, denied, invalidation) + ] + await session.commit() + + assert [event.event_type for event in stored] == [ + "SensitiveAuthorizationAllowed", + "SensitiveAuthorizationDenied", + "AuthorityInvalidationRequested", + ] + assert all(event.occurred_at is not None for event in stored) + for event in stored: + assert event.event_domain == "authority" + assert event.event_version == 1 + assert event.external_subject is None + assert event.external_issuer is None + assert event.actor_roles == [] + assert event.claim_snapshot == {} + assert event.event_payload == {} + assert event.auth_source == "local_authority" + assert event.is_dev_auth is False + assert stored[0].idempotency_reference is None + assert UUID(stored[2].idempotency_reference) == future_idempotency + + +async def test_legacy_writer_and_reader_remain_compatible(audit_factory) -> None: + event = AuditEvent( + id=str(uuid4()), + entity_type="task", + entity_id=str(uuid4()), + event_type="task_created", + from_status=None, + to_status="draft", + actor_id="legacy-actor", + external_subject="opaque-subject", + external_issuer="https://issuer.example.test", + actor_roles=["project_manager"], + claim_snapshot={"bounded": True}, + auth_source="verified_token", + is_dev_auth=False, + event_payload={"source_type": "manual"}, + ) + async with audit_factory() as session: + repository = AuditRepository(session) + stored = await repository.add_audit_event(event) + await session.commit() + listed = await repository.list_audit_events("task", event.entity_id) + + assert listed == [stored] + assert stored.event_domain == "legacy_lifecycle" + assert stored.event_version is None + assert stored.occurred_at is None + assert stored.event_payload == {"source_type": "manual"} + + +async def test_authority_event_rollback_leaves_no_row(audit_factory) -> None: + value = _authority_input(AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED) + async with audit_factory() as session: + await AuditService(session).add_authority_event(value) + await session.rollback() + async with audit_factory() as session: + assert await session.get(AuditEvent, str(value.event_id)) is None + + +async def test_database_rejects_malformed_and_mutated_audit_rows(audit_factory) -> None: + value = _authority_input(AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED) + legacy_id = str(uuid4()) + async with audit_factory() as session: + await AuditService(session).add_authority_event(value) + await AuditRepository(session).add_audit_event( + AuditEvent( + id=legacy_id, + entity_type="task", + entity_id=str(uuid4()), + event_type="task_created", + actor_id="legacy-actor", + external_subject="opaque-subject", + external_issuer="https://issuer.example.test", + actor_roles=[], + claim_snapshot={}, + auth_source="verified_token", + is_dev_auth=False, + event_payload={}, + ) + ) + await session.commit() + + async with audit_factory() as session: + for event_id in (str(value.event_id), legacy_id): + with pytest.raises(DBAPIError, match="append-only"): + await session.execute( + text("update audit_events set reason = 'changed' where id = :id"), + {"id": event_id}, + ) + await session.rollback() + with pytest.raises(DBAPIError, match="append-only"): + await session.execute( + text("delete from audit_events where id = :id"), + {"id": event_id}, + ) + await session.rollback() + with pytest.raises(DBAPIError, match="append-only"): + await session.execute(text("truncate table audit_events cascade")) + await session.rollback() + rows = ( + await session.execute( + text( + "select id, reason, event_domain from audit_events " + "where id in (:authority_id, :legacy_id) order by id" + ), + {"authority_id": str(value.event_id), "legacy_id": legacy_id}, + ) + ).all() + assert {(row.id, row.reason, row.event_domain) for row in rows} == { + (str(value.event_id), None, "authority"), + (legacy_id, None, "legacy_lifecycle"), + } + + with pytest.raises(IntegrityError): + await session.execute( + text( + "insert into audit_events " + "(id, entity_type, entity_id, event_type, actor_id, actor_roles, " + "claim_snapshot, auth_source, is_dev_auth, event_payload, event_domain) " + "values (:id, 'actor', :entity_id, 'SensitiveAuthorizationAllowed', " + "'legacy', '[]', '{}', 'local_authority', false, '{}', 'authority')" + ), + {"id": str(uuid4()), "entity_id": str(uuid4())}, + ) + await session.rollback() diff --git a/backend/tests/test_tasks.py b/backend/tests/test_tasks.py index 4701faef..ee1a1b81 100644 --- a/backend/tests/test_tasks.py +++ b/backend/tests/test_tasks.py @@ -7,13 +7,14 @@ from datetime import UTC, datetime, timedelta from decimal import Decimal from pathlib import Path +from unittest.mock import AsyncMock, MagicMock from uuid import uuid4 import pytest from alembic import command from alembic.config import Config from httpx import ASGITransport, AsyncClient -from sqlalchemy import inspect, select +from sqlalchemy import inspect, select, text from sqlalchemy.dialects import postgresql from sqlalchemy.exc import IntegrityError from sqlalchemy.schema import CreateIndex @@ -57,6 +58,47 @@ from app.schemas.auth import ActorContext +async def test_task_repository_delegates_audit_persistence() -> None: + """Keep legacy task audit methods as same-session shared-writer adapters.""" + session = MagicMock() + repository = TaskRepository(session) + event = MagicMock(spec=AuditEvent) + persisted = MagicMock(spec=AuditEvent) + listed = [persisted] + repository._audit_repository.add_audit_event = AsyncMock(return_value=persisted) + repository._audit_repository.list_audit_events = AsyncMock(return_value=listed) + + assert repository._audit_repository._session is session + assert await repository.add_audit_event(event) is persisted + assert await repository.list_audit_events("task", "task-1") is listed + repository._audit_repository.add_audit_event.assert_awaited_once_with(event) + repository._audit_repository.list_audit_events.assert_awaited_once_with( + "task", "task-1" + ) + + +async def delete_audit_fixture_as_owner(session, event_id: str) -> None: + """Construct missing-evidence corruption under explicit test-owner custody.""" + await session.execute(text("lock table audit_events in access exclusive mode")) + await session.execute( + text( + "alter table audit_events disable trigger " + "audit_events_reject_update_delete" + ) + ) + await session.execute( + text("delete from audit_events where id = :event_id"), + {"event_id": event_id}, + ) + await session.execute( + text( + "alter table audit_events enable trigger " + "audit_events_reject_update_delete" + ) + ) + await session.commit() + + @pytest.fixture def task_database_env( monkeypatch: pytest.MonkeyPatch, @@ -4821,8 +4863,7 @@ async def test_manual_checker_run_cannot_bypass_failed_automatic_gate( ) assert queued_run is not None assert lock_audit is not None - await session.delete(lock_audit) - await session.commit() + await delete_audit_fixture_as_owner(session, lock_audit.id) with pytest.raises(CheckerExecutionBlocked): run_pre_review_gate.run( @@ -5075,8 +5116,7 @@ async def test_queued_gate_fails_closed_when_lock_audit_is_missing( ) assert queued_run is not None assert lock_audit is not None - await session.delete(lock_audit) - await session.commit() + await delete_audit_fixture_as_owner(session, lock_audit.id) with pytest.raises(CheckerExecutionBlocked): run_pre_review_gate.run( diff --git a/docs/architecture_data_model.md b/docs/architecture_data_model.md index 07c11827..1223479f 100644 --- a/docs/architecture_data_model.md +++ b/docs/architecture_data_model.md @@ -1497,16 +1497,26 @@ Fields: Audit events are append-only. -`actor_roles` and `claim_snapshot` are legacy-only columns. New authority events -must leave them empty and use only the bounded actor, matched grant/permission, -scope, resource, reason, and before/after fields. Raw claims and unnecessary -profile data are prohibited. +Authority evidence extends the same row with `event_domain`, `event_version`, +database-owned `occurred_at`, request/correlation IDs, an actor-reference +namespace, and optional bounded target actor, matched grant, permission, +project, resource, denial, invalidation, idempotency-reference, and shallow +before/after fact fields. It does not create a parallel event table. + +`actor_roles`, `claim_snapshot`, and `event_payload` are legacy-only data +surfaces. Authority rows keep them at `[]`, `{}`, and `{}` respectively, use +`auth_source = local_authority`, and leave external issuer, external subject, +and lifecycle status fields null. They never store tokens, claims, emails, +request bodies, issuer URLs, key material, or policy bodies. Typed validation +and database constraints reject mixed legacy/authority shapes. v0.1 audit storage is the existing Workstream `audit_events` ledger. Task Lifecycle events and authority events share the canonical audit repository so operators can reconstruct why an actor was allowed or denied. Authority events record bounded actor, matched grant/permission, scope, resource, reason, and -before/after facts without raw claims or unnecessary profile data. +before/after facts without raw claims or unnecessary profile data. The shared +repository participates in its caller's transaction and does not commit or +open an independent session. ## Required Invariants diff --git a/docs/operations_authorization_service.md b/docs/operations_authorization_service.md index 3b9915c4..60dfd9e0 100644 --- a/docs/operations_authorization_service.md +++ b/docs/operations_authorization_service.md @@ -440,6 +440,37 @@ metrics/configuration evidence for the normal security incident period. Never use token, subject, email, `jti`, raw URL, key material, or unbounded resource IDs as metric labels. +## Authority Audit Custody + +Authority decisions and invalidation requests use the shared `audit_events` +ledger. Database triggers reject normal `UPDATE`, `DELETE`, and `TRUNCATE` for +both lifecycle and authority rows. There is no runtime flag, session setting, +or application-user bypass. Authority timestamps come from the database, and +the application writer joins the caller's transaction so evidence cannot +commit separately from the authority mutation it explains. + +Production must run Workstream with a dedicated non-owner database role that +has no DDL, trigger-management, table-owner, superuser, or bypass privileges. +Release verification must fail if the application credential owns +`audit_events` or can disable its triggers. Schema migration credentials are +separate and unavailable to the running service. + +Owner-level maintenance is exceptional and requires an approved change record, +an outage or writer drain, and an explicit list of rows and purpose. The +database owner must: + +1. begin one transaction and take `ACCESS EXCLUSIVE` lock on `audit_events`; +2. record the pre-change row count and trigger state without exporting private data; +3. disable only the named mutation trigger needed for the approved operation; +4. perform only the listed maintenance statements and verify affected IDs/counts; +5. re-enable the trigger before commit and prove all three audit triggers enabled; +6. retain redacted change evidence and return credentials to controlled storage. + +Do not use owner maintenance to revise authority history, erase a denial, or +fabricate evidence. Normal migration downgrade refuses while authority rows +exist; destructive cleanup requires a separately reviewed retention or legal +procedure and is not an application operation. + ## Incident Response For suspected authority misuse: From 6fbb1f8c13379eb7b7e121eab9d9b8c083f25e04 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 11:40:18 +0100 Subject: [PATCH 05/22] fix(auth): close authority evidence privacy gaps --- .../versions/0018_authority_audit_evidence.py | 124 +++++----- backend/app/modules/audit/repository.py | 24 +- backend/app/modules/audit/schemas.py | 109 ++++++--- backend/app/modules/audit/service.py | 42 +--- backend/tests/test_alembic.py | 198 ++++++++-------- backend/tests/test_audit.py | 214 ++++++++++++++++-- 6 files changed, 445 insertions(+), 266 deletions(-) diff --git a/backend/alembic/versions/0018_authority_audit_evidence.py b/backend/alembic/versions/0018_authority_audit_evidence.py index 05d142d1..10db9ed7 100644 --- a/backend/alembic/versions/0018_authority_audit_evidence.py +++ b/backend/alembic/versions/0018_authority_audit_evidence.py @@ -72,32 +72,27 @@ def upgrade() -> None: op.add_column("audit_events", column) op.alter_column("audit_events", "external_subject", existing_type=sa.String(200), nullable=True) op.alter_column("audit_events", "external_issuer", existing_type=sa.String(200), nullable=True) + op.execute("""create function authority_facts_are_safe(facts json) returns boolean language sql immutable strict as $$ + select json_typeof(facts) = 'object' and not exists (select 1 from json_each(facts) item where + item.key not in ('status','subject_kind','provisioning_method','link_status','role_id','scope_type','scope_id','effective','qualification_status','decision_code','target_status') + or json_typeof(item.value) not in ('string','boolean','null') + or (item.key = 'effective' and json_typeof(item.value) not in ('boolean','null')) + or (item.key <> 'effective' and json_typeof(item.value) not in ('string','null')) + or (json_typeof(item.value) = 'string' and ((item.value #>> '{}') ~ '^eyJ' or (item.key in ('role_id','scope_id') and (item.value #>> '{}') !~ '^[A-Za-z0-9][A-Za-z0-9_.:-]{0,119}$') or (item.key not in ('role_id','scope_id') and (item.value #>> '{}') !~ '^[a-z][a-z0-9_]{0,79}$')))) $$""") event_tokens = ", ".join(f"'{value}'" for value in AUTHORITY_EVENT_TYPES) op.create_check_constraint( "domain_shape", "audit_events", """ - (event_domain = 'legacy_lifecycle' and event_version is null and occurred_at is null - and actor_ref_kind is null and request_id is null and correlation_id is null - and target_actor_ref_kind is null and target_actor_ref is null - and matched_grant_id is null and permission_id is null and project_id is null - and resource_type is null and resource_id is null - and target_ref_kind is null and target_ref_id is null and denial_code is null - and idempotency_reference is null and invalidation_cause_event_id is null - and invalidation_target_kind is null and invalidation_target_ref is null - and before_facts is null and after_facts is null - and external_subject is not null and external_issuer is not null) - or - (event_domain = 'authority' and event_version = 1 and occurred_at is not null - and actor_ref_kind in ('legacy_actor', 'actor_profile', 'system_principal') - and request_id is not null and correlation_id is not null - and from_status is null and to_status is null - and (reason is null or length(reason) <= 1000) - and external_subject is null and external_issuer is null - and actor_roles::jsonb = '[]'::jsonb and claim_snapshot::jsonb = '{}'::jsonb - and auth_source = 'local_authority' and is_dev_auth = false - and event_payload::jsonb = '{}'::jsonb) + (event_domain = 'legacy_lifecycle' and event_version is null and occurred_at is null and actor_ref_kind is null and request_id is null and correlation_id is null + and target_actor_ref_kind is null and target_actor_ref is null and matched_grant_id is null and permission_id is null and project_id is null + and resource_type is null and resource_id is null and target_ref_kind is null and target_ref_id is null and denial_code is null + and idempotency_reference is null and invalidation_cause_event_id is null and invalidation_target_kind is null and invalidation_target_ref is null + and before_facts is null and after_facts is null and external_subject is not null and external_issuer is not null) + or (event_domain = 'authority' and event_version = 1 and occurred_at is not null and actor_ref_kind in ('legacy_actor','actor_profile','system_principal') + and request_id is not null and correlation_id is not null and from_status is null and to_status is null and (reason is null or length(reason) <= 1000) + and external_subject is null and external_issuer is null and actor_roles::jsonb = '[]'::jsonb and claim_snapshot::jsonb = '{}'::jsonb and auth_source = 'local_authority' and is_dev_auth = false and event_payload::jsonb = '{}'::jsonb) """, ) op.create_check_constraint( @@ -109,38 +104,43 @@ def upgrade() -> None: "reference_pairs", "audit_events", """ - (target_actor_ref_kind is null) = (target_actor_ref is null) - and (resource_type is null) = (resource_id is null) - and (target_ref_kind is null) = (target_ref_id is null) - and (invalidation_target_kind is null) = (invalidation_target_ref is null) + (target_actor_ref_kind is null) = (target_actor_ref is null) and (resource_type is null) = (resource_id is null) + and (target_ref_kind is null) = (target_ref_id is null) and (invalidation_target_kind is null) = (invalidation_target_ref is null) + and (invalidation_cause_event_id is null or invalidation_cause_event_id <> id) """, ) op.create_check_constraint( - "fact_bounds", + "authority_privacy_bounds", "audit_events", """ - (before_facts is null or - (json_typeof(before_facts) = 'object' and octet_length(before_facts::text) <= 4096)) - and (after_facts is null or - (json_typeof(after_facts) = 'object' and octet_length(after_facts::text) <= 4096)) + event_domain <> 'authority' or ((((actor_ref_kind in ('legacy_actor','actor_profile') and actor_id ~ '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$') + or (actor_ref_kind = 'system_principal' and actor_id ~ '^[a-z][a-z0-9_]*(:[a-z][a-z0-9_-]*)+$')) + and (target_actor_ref is null or ((target_actor_ref_kind in ('legacy_actor','actor_profile') and target_actor_ref ~ '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$') + or (target_actor_ref_kind = 'system_principal' and target_actor_ref ~ '^[a-z][a-z0-9_]*(:[a-z][a-z0-9_-]*)+$'))) + and entity_id ~ '^[A-Za-z0-9][A-Za-z0-9_.:-]{0,35}$' and concat_ws('',matched_grant_id,permission_id,project_id,resource_type,resource_id,target_ref_kind,target_ref_id,denial_code,invalidation_target_kind,invalidation_target_ref) !~ '[/@[:space:]]' + and (reason is null or reason ~ '^[A-Za-z0-9][A-Za-z0-9 _.,:;()!?-]*$'))) """, ) + op.create_check_constraint( + "fact_bounds", + "audit_events", + "(before_facts is null or (authority_facts_are_safe(before_facts) and octet_length(before_facts::text) <= 4096)) and (after_facts is null or (authority_facts_are_safe(after_facts) and octet_length(after_facts::text) <= 4096))", + ) + op.create_foreign_key( + "fk_audit_events_invalidation_cause", + "audit_events", + "audit_events", + ["invalidation_cause_event_id"], + ["id"], + ) op.create_check_constraint( "foundation_shapes", "audit_events", """ - event_domain <> 'authority' - or event_type not in ('SensitiveAuthorizationAllowed', - 'SensitiveAuthorizationDenied', 'AuthorityInvalidationRequested') - or (event_type = 'SensitiveAuthorizationAllowed' and permission_id is not null - and denial_code is null and invalidation_cause_event_id is null - and invalidation_target_kind is null) - or (event_type = 'SensitiveAuthorizationDenied' and permission_id is not null - and denial_code is not null and invalidation_cause_event_id is null - and invalidation_target_kind is null and idempotency_reference is null) - or (event_type = 'AuthorityInvalidationRequested' - and invalidation_cause_event_id is not null - and invalidation_target_kind is not null and denial_code is null) + event_domain <> 'authority' or event_type not in ('SensitiveAuthorizationAllowed','SensitiveAuthorizationDenied','AuthorityInvalidationRequested') + or (event_type = 'SensitiveAuthorizationAllowed' and permission_id is not null and denial_code is null and invalidation_cause_event_id is null and invalidation_target_kind is null) + or (event_type = 'SensitiveAuthorizationDenied' and permission_id is not null and denial_code is not null and invalidation_cause_event_id is null and invalidation_target_kind is null and idempotency_reference is null) + or (event_type = 'AuthorityInvalidationRequested' and invalidation_cause_event_id is not null and invalidation_target_kind is not null and denial_code is null) """, ) for name, fields in ( @@ -152,35 +152,16 @@ def upgrade() -> None: ): op.create_index(name, "audit_events", fields) - op.execute(""" - create function set_authority_audit_database_time() returns trigger language plpgsql as $$ - begin - if new.event_domain = 'authority' then - new.occurred_at = statement_timestamp(); - else - new.occurred_at = null; - end if; - return new; - end $$ - """) - op.execute(""" - create trigger audit_events_set_authority_time before insert on audit_events - for each row execute function set_authority_audit_database_time() - """) - op.execute(""" - create function reject_audit_event_mutation() returns trigger language plpgsql as $$ - begin - raise exception 'audit events are append-only' using errcode = '55000'; - end $$ - """) - op.execute(""" - create trigger audit_events_reject_update_delete before update or delete on audit_events - for each row execute function reject_audit_event_mutation() - """) - op.execute(""" - create trigger audit_events_reject_truncate before truncate on audit_events - for each statement execute function reject_audit_event_mutation() - """) + op.execute("""create function set_authority_audit_database_time() returns trigger language plpgsql as $$ begin + if new.event_domain = 'authority' then + if new.invalidation_cause_event_id is not null and not exists (select 1 from audit_events where id = new.invalidation_cause_event_id and event_domain = 'authority') then + raise exception 'invalid authority invalidation cause' using errcode = '23503'; end if; + new.occurred_at = statement_timestamp(); + else new.occurred_at = null; end if; return new; end $$""") + op.execute("create trigger audit_events_set_authority_time before insert on audit_events for each row execute function set_authority_audit_database_time()") + op.execute("""create function reject_audit_event_mutation() returns trigger language plpgsql as $$ begin raise exception 'audit events are append-only' using errcode = '55000'; end $$""") + op.execute("create trigger audit_events_reject_update_delete before update or delete on audit_events for each row execute function reject_audit_event_mutation()") + op.execute("create trigger audit_events_reject_truncate before truncate on audit_events for each statement execute function reject_audit_event_mutation()") def downgrade() -> None: @@ -209,14 +190,17 @@ def downgrade() -> None: "ix_audit_events_request_id", ): op.drop_index(name, table_name="audit_events") + op.drop_constraint("fk_audit_events_invalidation_cause", "audit_events", type_="foreignkey") for name in ( "foundation_shapes", "fact_bounds", + "authority_privacy_bounds", "reference_pairs", "authority_tokens", "domain_shape", ): op.drop_constraint(name, "audit_events", type_="check") + op.execute("drop function authority_facts_are_safe(json)") for column in reversed( ( "event_version", "occurred_at", "actor_ref_kind", "request_id", "correlation_id", diff --git a/backend/app/modules/audit/repository.py b/backend/app/modules/audit/repository.py index 25b97688..1d9a12f2 100644 --- a/backend/app/modules/audit/repository.py +++ b/backend/app/modules/audit/repository.py @@ -30,14 +30,32 @@ async def add_audit_event(self, event: AuditEvent) -> AuditEvent: Returns: Persisted audit event model. """ + if event.event_domain == "authority": + raise ValueError("authority events require the typed audit service") + return await self._persist(event) + + async def _add_validated_authority_event(self, event: AuditEvent) -> AuditEvent: + """Persist an authority event already validated by AuditService.""" + if event.event_domain != "authority": + raise ValueError("expected authority audit event") + return await self._persist(event) + + async def get_authority_event(self, event_id: str) -> AuditEvent | None: + """Return one authority cause visible in the caller transaction.""" + return await self._session.scalar( + select(AuditEvent).where( + AuditEvent.id == event_id, AuditEvent.event_domain == "authority" + ) + ) + + async def _persist(self, event: AuditEvent) -> AuditEvent: + """Flush one event without taking transaction ownership.""" self._session.add(event) await self._session.flush() await self._session.refresh(event) return event - async def list_audit_events( - self, entity_type: str, entity_id: str - ) -> Sequence[AuditEvent]: + async def list_audit_events(self, entity_type: str, entity_id: str) -> Sequence[AuditEvent]: """List events for one entity in database creation order.""" result = await self._session.execute( select(AuditEvent) diff --git a/backend/app/modules/audit/schemas.py b/backend/app/modules/audit/schemas.py index dda5441f..2220e69b 100644 --- a/backend/app/modules/audit/schemas.py +++ b/backend/app/modules/audit/schemas.py @@ -5,12 +5,49 @@ from enum import StrEnum import json import re -from typing import Self +from typing import Annotated, Self from uuid import UUID, uuid4 from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator _FACT_KEY = re.compile(r"^[a-z][a-z0-9_]{0,63}$") +_SAFE_REF = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_.:-]{0,119}$") +_SYSTEM_REF = re.compile(r"^[a-z][a-z0-9_]*(?::[a-z][a-z0-9_-]*)+$") +_SAFE_REASON = re.compile(r"^[A-Za-z0-9][A-Za-z0-9 _.,:;()!?-]{0,999}$") +_FACT_BOOL_KEYS = frozenset({"effective"}) +_FACT_ID_KEYS = frozenset({"role_id", "scope_id"}) +_FACT_TOKEN = re.compile(r"^[a-z][a-z0-9_]{0,79}$") +SafeRef100 = Annotated[str, Field(max_length=100, pattern=_SAFE_REF.pattern)] +SafeRef36 = Annotated[str, Field(max_length=36, pattern=_SAFE_REF.pattern)] +SafeToken = Annotated[str, Field(max_length=80, pattern=r"^[a-z][a-z0-9_.:-]*$")] +SafeToken32 = Annotated[str, Field(max_length=32, pattern=r"^[a-z][a-z0-9_.:-]*$")] + + +def _is_uuid(value: str) -> bool: + """Return whether a value is a canonical local UUID.""" + try: + return str(UUID(value)) == value + except ValueError: + return False + + +def _safe_fact_value(key: str, value: object) -> bool: + """Accept only typed booleans or bounded local tokens in state facts.""" + if value is None: + return True + if key in _FACT_BOOL_KEYS: + return type(value) is bool + pattern = _SAFE_REF if key in _FACT_ID_KEYS else _FACT_TOKEN + return ( + isinstance(value, str) + and not value.startswith("eyJ") + and pattern.fullmatch(value) is not None + ) + + +def _safe_actor_ref(kind: ActorReferenceKind, value: str) -> bool: + """Validate one local or registered system actor reference.""" + return bool(_SYSTEM_REF.fullmatch(value)) if kind == ActorReferenceKind.SYSTEM_PRINCIPAL else _is_uuid(value) class AuthorityEventType(StrEnum): @@ -51,11 +88,10 @@ class AuthorityEventType(StrEnum): {"status", "role_id", "scope_type", "scope_id", "effective", "qualification_status"} ), ), - **dict.fromkeys( - _EVENT_TYPES[17:19], frozenset({"status", "decision_code", "effective"}) - ), + **dict.fromkeys(_EVENT_TYPES[17:19], frozenset({"status", "decision_code", "effective"})), _EVENT_TYPES[19]: frozenset({"status", "target_status", "effective"}), } +_ALL_FACT_KEYS = frozenset().union(*_EVENT_FACT_KEYS.values()) class ActorReferenceKind(StrEnum): @@ -73,39 +109,37 @@ class AuthorityAuditEventInput(BaseModel): event_id: UUID = Field(default_factory=uuid4) event_type: AuthorityEventType - entity_type: str = Field(min_length=1, max_length=80, pattern=r"^[a-z][a-z0-9_.:-]*$") - entity_id: str = Field(min_length=1, max_length=36) + entity_type: SafeToken + entity_id: SafeRef36 actor_ref_kind: ActorReferenceKind - actor_ref: str = Field(min_length=1, max_length=100) + actor_ref: SafeRef100 request_id: UUID correlation_id: UUID target_actor_ref_kind: ActorReferenceKind | None = None - target_actor_ref: str | None = Field(default=None, min_length=1, max_length=100) - matched_grant_id: str | None = Field(default=None, min_length=1, max_length=100) - permission_id: str | None = Field( - default=None, min_length=1, max_length=120, pattern=r"^[a-z][a-z0-9_.:-]*$" - ) - project_id: str | None = Field(default=None, min_length=1, max_length=36) - resource_type: str | None = Field( - default=None, min_length=1, max_length=80, pattern=r"^[a-z][a-z0-9_.:-]*$" - ) - resource_id: str | None = Field(default=None, min_length=1, max_length=100) - target_ref_kind: str | None = Field( - default=None, min_length=1, max_length=32, pattern=r"^[a-z][a-z0-9_.:-]*$" - ) - target_ref_id: str | None = Field(default=None, min_length=1, max_length=100) - reason: str | None = Field(default=None, max_length=1000) - denial_code: str | None = Field( - default=None, min_length=1, max_length=80, pattern=r"^[a-z][a-z0-9_]*$" - ) + target_actor_ref: SafeRef100 | None = None + matched_grant_id: SafeRef100 | None = None + permission_id: Annotated[str, Field(max_length=120, pattern=r"^[a-z][a-z0-9_.:-]*$")] | None = None + project_id: SafeRef36 | None = None + resource_type: SafeToken | None = None + resource_id: SafeRef100 | None = None + target_ref_kind: SafeToken32 | None = None + target_ref_id: SafeRef100 | None = None + reason: str | None = Field(default=None, max_length=1000, pattern=_SAFE_REASON.pattern) + denial_code: Annotated[str, Field(max_length=80, pattern=r"^[a-z][a-z0-9_]*$")] | None = None idempotency_reference: UUID | None = None invalidation_cause_event_id: UUID | None = None - invalidation_target_kind: str | None = Field( - default=None, min_length=1, max_length=32, pattern=r"^[a-z][a-z0-9_.:-]*$" - ) - invalidation_target_ref: str | None = Field(default=None, min_length=1, max_length=100) - before_facts: dict[str, str | int | bool | None] | None = None - after_facts: dict[str, str | int | bool | None] | None = None + invalidation_target_kind: SafeToken32 | None = None + invalidation_target_ref: SafeRef100 | None = None + before_facts: dict[str, str | bool | None] | None = None + after_facts: dict[str, str | bool | None] | None = None + + @model_validator(mode="before") + @classmethod + def reject_unknown_fields_without_echo(cls, value: object) -> object: + """Reject unknown fields before Pydantic can retain their values.""" + if isinstance(value, dict) and value.keys() - cls.model_fields.keys(): + raise TypeError("unexpected authority audit fields") + return value @field_validator("before_facts", "after_facts", mode="before") @classmethod @@ -113,16 +147,18 @@ def validate_facts(cls, value: object) -> object: """Reject nested, excessive, or oversized state facts.""" if value is None: return None - if not isinstance(value, dict) or any( - not isinstance(key, str) for key in value - ): + if not isinstance(value, dict) or any(not isinstance(key, str) for key in value): raise ValueError("invalid authority state facts") if len(value) > 16 or any(_FACT_KEY.fullmatch(key) is None for key in value): raise ValueError("invalid authority state facts") + if not set(value).issubset(_ALL_FACT_KEYS): + raise ValueError("authority state facts are not allowed for this event") if any(isinstance(item, (dict, list, tuple, set)) for item in value.values()): raise ValueError("invalid authority state facts") if len(json.dumps(value, separators=(",", ":"), ensure_ascii=False).encode()) > 4096: raise ValueError("authority state facts exceed size limit") + if any(not _safe_fact_value(key, item) for key, item in value.items()): + raise ValueError("invalid authority state fact value") return value @model_validator(mode="after") @@ -136,6 +172,11 @@ def validate_shape(self) -> Self: ): if (left is None) != (right is None): raise ValueError("authority reference fields must be paired") + references = ((self.actor_ref_kind, self.actor_ref), (self.target_actor_ref_kind, self.target_actor_ref)) + if any(kind is not None and not _safe_actor_ref(kind, reference or "") for kind, reference in references): + raise ValueError("actor references must use canonical local identifiers") + if self.invalidation_cause_event_id == self.event_id: + raise ValueError("invalidation cannot reference its own event") invalidation = self.invalidation_cause_event_id is not None or self.invalidation_target_kind allowed_fact_keys = _EVENT_FACT_KEYS[self.event_type] if any( diff --git a/backend/app/modules/audit/service.py b/backend/app/modules/audit/service.py index 6f2945f2..db97c023 100644 --- a/backend/app/modules/audit/service.py +++ b/backend/app/modules/audit/service.py @@ -16,50 +16,24 @@ def __init__(self, session: AsyncSession) -> None: async def add_authority_event(self, value: AuthorityAuditEventInput) -> AuditEvent: """Persist one validated authority event without committing its transaction.""" + cause_id = value.invalidation_cause_event_id + if cause_id is not None and await self._repository.get_authority_event(str(cause_id)) is None: + raise ValueError("invalidation cause must be an existing authority event") + fields = value.model_dump(mode="json") + fields["id"] = fields.pop("event_id") + fields["actor_id"] = fields.pop("actor_ref") event = AuditEvent( - id=str(value.event_id), - entity_type=value.entity_type, - entity_id=value.entity_id, - event_type=value.event_type.value, + **fields, from_status=None, to_status=None, - actor_id=value.actor_ref, external_subject=None, external_issuer=None, actor_roles=[], claim_snapshot={}, auth_source="local_authority", is_dev_auth=False, - reason=value.reason, event_payload={}, event_domain="authority", event_version=1, - actor_ref_kind=value.actor_ref_kind.value, - request_id=str(value.request_id), - correlation_id=str(value.correlation_id), - target_actor_ref_kind=( - value.target_actor_ref_kind.value if value.target_actor_ref_kind else None - ), - target_actor_ref=value.target_actor_ref, - matched_grant_id=value.matched_grant_id, - permission_id=value.permission_id, - project_id=value.project_id, - resource_type=value.resource_type, - resource_id=value.resource_id, - target_ref_kind=value.target_ref_kind, - target_ref_id=value.target_ref_id, - denial_code=value.denial_code, - idempotency_reference=( - str(value.idempotency_reference) if value.idempotency_reference else None - ), - invalidation_cause_event_id=( - str(value.invalidation_cause_event_id) - if value.invalidation_cause_event_id - else None - ), - invalidation_target_kind=value.invalidation_target_kind, - invalidation_target_ref=value.invalidation_target_ref, - before_facts=value.before_facts, - after_facts=value.after_facts, ) - return await self._repository.add_audit_event(event) + return await self._repository._add_validated_authority_event(event) diff --git a/backend/tests/test_alembic.py b/backend/tests/test_alembic.py index 5d9d6b9a..6cfc335b 100644 --- a/backend/tests/test_alembic.py +++ b/backend/tests/test_alembic.py @@ -17,6 +17,7 @@ from app.adapters.auth.dev import actor_id_from_external_identity + def test_alembic_upgrade_and_downgrade(isolated_database_env: str, migration_lock) -> None: project_root = Path(__file__).resolve().parents[1] config = Config(str(project_root / "alembic.ini")) @@ -143,7 +144,9 @@ def test_post_submit_policy_upgrade_blocks_pre_provenance_runtime_rows( config = Config(str(project_root / "alembic.ini")) config.set_main_option("script_location", str(project_root / "alembic")) - ids = {name: str(uuid4()) for name in ("project", "guide", "policy", "task", "submission", "run")} + ids = { + name: str(uuid4()) for name in ("project", "guide", "policy", "task", "submission", "run") + } with migration_lock(): try: command.downgrade(config, "base") @@ -287,9 +290,7 @@ def test_artifact_foundation_enforces_immutable_facts_and_guarded_downgrade( command.upgrade(config, "0016_artifact_domain") assert all( count == 0 - for count in asyncio.run( - _artifact_table_counts(isolated_database_env) - ).values() + for count in asyncio.run(_artifact_table_counts(isolated_database_env)).values() ) finally: command.downgrade(config, "base") @@ -313,20 +314,14 @@ def test_api_rate_control_schema_preserves_domain_and_guards_downgrade( command.upgrade(config, "0015_post_submit_correction") asyncio.run(_seed_artifact_prior_head(isolated_database_env, project_id)) command.upgrade(config, "0016_artifact_domain") - before = asyncio.run( - _artifact_prior_head_project(isolated_database_env, project_id) - ) + before = asyncio.run(_artifact_prior_head_project(isolated_database_env, project_id)) artifact_before = asyncio.run( _seed_and_fetch_0016_artifact(isolated_database_env, artifact_id) ) command.upgrade(config, "0017_api_controls") - after = asyncio.run( - _artifact_prior_head_project(isolated_database_env, project_id) - ) - artifact_after = asyncio.run( - _fetch_0016_artifact(isolated_database_env, artifact_id) - ) + after = asyncio.run(_artifact_prior_head_project(isolated_database_env, project_id)) + artifact_after = asyncio.run(_fetch_0016_artifact(isolated_database_env, artifact_id)) schema = asyncio.run(_api_rate_control_schema(isolated_database_env)) asyncio.run(_assert_api_rate_control_guards(isolated_database_env, digest)) @@ -365,9 +360,7 @@ def guarded_downgrade() -> None: refused_state = asyncio.run(_api_rate_control_state(isolated_database_env)) asyncio.run(_clear_api_rate_controls(isolated_database_env)) command.downgrade(config, "0016_artifact_domain") - downgraded_state = asyncio.run( - _api_rate_control_state(isolated_database_env) - ) + downgraded_state = asyncio.run(_api_rate_control_state(isolated_database_env)) command.upgrade(config, "0017_api_controls") finally: asyncio.run(_clear_api_rate_controls(isolated_database_env)) @@ -425,40 +418,26 @@ def test_authority_audit_schema_preserves_legacy_and_guards_downgrade( try: command.downgrade(config, "base") command.upgrade(config, "0017_api_controls") - before = asyncio.run( - _seed_and_fetch_legacy_audit(isolated_database_env, legacy_id) - ) + before = asyncio.run(_seed_and_fetch_legacy_audit(isolated_database_env, legacy_id)) command.upgrade(config, "0018_authority_audit_evidence") after = asyncio.run(_fetch_audit_row(isolated_database_env, legacy_id)) schema = asyncio.run(_authority_audit_schema(isolated_database_env)) occurred_at = asyncio.run( - _insert_authority_audit_fixture( - isolated_database_env, authority_id - ) + _insert_authority_audit_fixture(isolated_database_env, authority_id) ) with pytest.raises(RuntimeError, match="non-empty authority audit"): command.downgrade(config, "0017_api_controls") refused = asyncio.run(_authority_audit_state(isolated_database_env)) - asyncio.run( - _remove_authority_audit_fixture( - isolated_database_env, authority_id - ) - ) + asyncio.run(_remove_authority_audit_fixture(isolated_database_env, authority_id)) command.downgrade(config, "0017_api_controls") downgraded = asyncio.run(_fetch_audit_row(isolated_database_env, legacy_id)) command.upgrade(config, "0018_authority_audit_evidence") - restored_schema = asyncio.run( - _authority_audit_schema(isolated_database_env) - ) + restored_schema = asyncio.run(_authority_audit_schema(isolated_database_env)) finally: - asyncio.run( - _remove_authority_audit_fixture( - isolated_database_env, authority_id - ) - ) + asyncio.run(_remove_authority_audit_fixture(isolated_database_env, authority_id)) command.downgrade(config, "base") command.upgrade(config, "head") @@ -496,11 +475,13 @@ def test_authority_audit_schema_preserves_legacy_and_guards_downgrade( "target_ref_kind:varchar:YES", }, "constraints": { + "ck_audit_events_authority_privacy_bounds", "ck_audit_events_authority_tokens", "ck_audit_events_domain_shape", "ck_audit_events_fact_bounds", "ck_audit_events_foundation_shapes", "ck_audit_events_reference_pairs", + "fk_audit_events_invalidation_cause", }, "indexes": { "ix_audit_events_actor_ref", @@ -515,6 +496,7 @@ def test_authority_audit_schema_preserves_legacy_and_guards_downgrade( "audit_events_set_authority_time", }, "functions": { + "authority_facts_are_safe", "reject_audit_event_mutation", "set_authority_audit_database_time", }, @@ -1012,7 +994,9 @@ async def _seed_pre_provenance_runtime_rows(database_url: str, ids: dict[str, st await engine.dispose() -async def _seed_pre_provenance_policies(connection, project_id: str, checker_policy_id: str) -> None: +async def _seed_pre_provenance_policies( + connection, project_id: str, checker_policy_id: str +) -> None: """Seed v0.1 guide policies required by locked task foreign keys.""" await connection.execute( text( @@ -1523,11 +1507,15 @@ async def _artifact_prior_head_project(database_url: str, project_id: str) -> di try: async with engine.connect() as connection: row = ( - await connection.execute( - text("select id, name, slug, status from projects where id = :id"), - {"id": project_id}, + ( + await connection.execute( + text("select id, name, slug, status from projects where id = :id"), + {"id": project_id}, + ) ) - ).mappings().one() + .mappings() + .one() + ) return dict(row) finally: await engine.dispose() @@ -1971,8 +1959,7 @@ async def _api_rate_control_schema(database_url: str) -> dict[str, set[str]]: ).scalars() return { "columns": { - f"{row.column_name}:{row.data_type}:{row.is_nullable}" - for row in columns + f"{row.column_name}:{row.data_type}:{row.is_nullable}" for row in columns }, "constraints": set(constraints), "indexes": set(indexes), @@ -1981,9 +1968,7 @@ async def _api_rate_control_schema(database_url: str) -> dict[str, set[str]]: await engine.dispose() -async def _seed_and_fetch_0016_artifact( - database_url: str, artifact_id: str -) -> dict[str, object]: +async def _seed_and_fetch_0016_artifact(database_url: str, artifact_id: str) -> dict[str, object]: """Seed one representative 0016 row and return its exact persisted value.""" engine = create_async_engine(database_url) try: @@ -2001,22 +1986,24 @@ async def _seed_and_fetch_0016_artifact( return await _fetch_0016_artifact(database_url, artifact_id) -async def _fetch_0016_artifact( - database_url: str, artifact_id: str -) -> dict[str, object]: +async def _fetch_0016_artifact(database_url: str, artifact_id: str) -> dict[str, object]: """Fetch the representative 0016 artifact-domain row.""" engine = create_async_engine(database_url) try: async with engine.begin() as connection: row = ( - await connection.execute( - text( - "select id, sha256, byte_count, media_type, " - "normalized_display_name from artifact_contents where id = :id" - ), - {"id": artifact_id}, + ( + await connection.execute( + text( + "select id, sha256, byte_count, media_type, " + "normalized_display_name from artifact_contents where id = :id" + ), + {"id": artifact_id}, + ) ) - ).mappings().one() + .mappings() + .one() + ) return dict(row) finally: await engine.dispose() @@ -2121,9 +2108,7 @@ async def _clear_api_rate_controls(database_url: str) -> None: await engine.dispose() -async def _seed_and_fetch_legacy_audit( - database_url: str, event_id: str -) -> dict[str, object]: +async def _seed_and_fetch_legacy_audit(database_url: str, event_id: str) -> dict[str, object]: """Insert one prior-head lifecycle event and return its legacy fields.""" engine = create_async_engine(database_url) try: @@ -2159,16 +2144,20 @@ async def _fetch_audit_row(database_url: str, event_id: str) -> dict[str, object ) domain = ", event_domain" if has_domain else "" row = ( - await connection.execute( - text( - "select id, entity_type, entity_id, event_type, from_status, " - "to_status, actor_id, external_subject, external_issuer, " - "actor_roles, claim_snapshot, auth_source, is_dev_auth, reason, " - f"event_payload{domain} from audit_events where id = :id" - ), - {"id": event_id}, + ( + await connection.execute( + text( + "select id, entity_type, entity_id, event_type, from_status, " + "to_status, actor_id, external_subject, external_issuer, " + "actor_roles, claim_snapshot, auth_source, is_dev_auth, reason, " + f"event_payload{domain} from audit_events where id = :id" + ), + {"id": event_id}, + ) ) - ).mappings().one() + .mappings() + .one() + ) return dict(row) finally: await engine.dispose() @@ -2177,26 +2166,45 @@ async def _fetch_audit_row(database_url: str, event_id: str) -> dict[str, object async def _authority_audit_schema(database_url: str) -> dict[str, object]: """Return the exact 0018 authority-audit schema surface.""" new_columns = { - "event_domain", "event_version", "occurred_at", "actor_ref_kind", - "request_id", "correlation_id", "target_actor_ref_kind", "target_actor_ref", - "matched_grant_id", "permission_id", "project_id", "resource_type", - "resource_id", "target_ref_kind", "target_ref_id", "denial_code", - "idempotency_reference", "invalidation_cause_event_id", - "invalidation_target_kind", "invalidation_target_ref", "before_facts", + "event_domain", + "event_version", + "occurred_at", + "actor_ref_kind", + "request_id", + "correlation_id", + "target_actor_ref_kind", + "target_actor_ref", + "matched_grant_id", + "permission_id", + "project_id", + "resource_type", + "resource_id", + "target_ref_kind", + "target_ref_id", + "denial_code", + "idempotency_reference", + "invalidation_cause_event_id", + "invalidation_target_kind", + "invalidation_target_ref", + "before_facts", "after_facts", } engine = create_async_engine(database_url) try: async with engine.begin() as connection: columns = ( - await connection.execute( - text( - "select column_name, udt_name, is_nullable, column_default " - "from information_schema.columns where table_schema = 'public' " - "and table_name = 'audit_events'" + ( + await connection.execute( + text( + "select column_name, udt_name, is_nullable, column_default " + "from information_schema.columns where table_schema = 'public' " + "and table_name = 'audit_events'" + ) ) ) - ).mappings().all() + .mappings() + .all() + ) by_name = {row["column_name"]: row for row in columns} constraints = set( ( @@ -2204,7 +2212,8 @@ async def _authority_audit_schema(database_url: str) -> dict[str, object]: text( "select conname from pg_constraint where " "conrelid = 'audit_events'::regclass " - "and conname like 'ck_audit_events_%'" + "and (conname like 'ck_audit_events_%' or " + "conname = 'fk_audit_events_invalidation_cause')" ) ) ).scalars() @@ -2237,7 +2246,7 @@ async def _authority_audit_schema(database_url: str) -> dict[str, object]: await connection.execute( text( "select proname from pg_proc where proname in " - "('reject_audit_event_mutation', " + "('authority_facts_are_safe', 'reject_audit_event_mutation', " "'set_authority_audit_database_time')" ) ) @@ -2263,9 +2272,7 @@ async def _authority_audit_schema(database_url: str) -> dict[str, object]: await engine.dispose() -async def _insert_authority_audit_fixture( - database_url: str, event_id: str -): +async def _insert_authority_audit_fixture(database_url: str, event_id: str): """Insert valid authority evidence while proving database-owned time.""" engine = create_async_engine(database_url) try: @@ -2303,19 +2310,14 @@ async def _authority_audit_state(database_url: str) -> dict[str, object]: text("select version_num from alembic_version") ), "authority_rows": await connection.scalar( - text( - "select count(*) from audit_events " - "where event_domain = 'authority'" - ) + text("select count(*) from audit_events where event_domain = 'authority'") ), } finally: await engine.dispose() -async def _remove_authority_audit_fixture( - database_url: str, event_id: str -) -> None: +async def _remove_authority_audit_fixture(database_url: str, event_id: str) -> None: """Perform explicit owner-only fixture cleanup under the documented lock.""" engine = create_async_engine(database_url) try: @@ -2330,22 +2332,14 @@ async def _remove_authority_audit_fixture( return await connection.execute(text("lock table audit_events in access exclusive mode")) await connection.execute( - text( - "alter table audit_events disable trigger " - "audit_events_reject_update_delete" - ) + text("alter table audit_events disable trigger audit_events_reject_update_delete") ) await connection.execute( - text( - "delete from audit_events where id = :id and event_domain = 'authority'" - ), + text("delete from audit_events where id = :id and event_domain = 'authority'"), {"id": event_id}, ) await connection.execute( - text( - "alter table audit_events enable trigger " - "audit_events_reject_update_delete" - ) + text("alter table audit_events enable trigger audit_events_reject_update_delete") ) finally: await engine.dispose() diff --git a/backend/tests/test_audit.py b/backend/tests/test_audit.py index d07ea984..a29d57e1 100644 --- a/backend/tests/test_audit.py +++ b/backend/tests/test_audit.py @@ -2,12 +2,14 @@ from pathlib import Path from uuid import UUID, uuid4 +from collections.abc import Mapping +import json from alembic import command from alembic.config import Config from pydantic import ValidationError import pytest -from sqlalchemy import text +from sqlalchemy import delete, text from sqlalchemy.exc import DBAPIError, IntegrityError from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine @@ -21,6 +23,25 @@ from app.modules.tasks.models import AuditEvent +def _assert_value_not_retained(value: object, forbidden: str, seen: set[int] | None = None) -> None: + """Traverse public exception state and prove rejected input is absent.""" + seen = seen or set() + if id(value) in seen: + return + seen.add(id(value)) + if isinstance(value, str): + assert forbidden not in value + elif isinstance(value, BaseException): + for item in (value.args, vars(value), value.__cause__, value.__context__): + _assert_value_not_retained(item, forbidden, seen) + elif isinstance(value, Mapping): + for key, item in value.items(): + _assert_value_not_retained((key, item), forbidden, seen) + elif isinstance(value, (list, tuple, set)): + for item in value: + _assert_value_not_retained(item, forbidden, seen) + + @pytest.fixture def audit_database_env(postgres_database_url: str, migration_lock) -> str: """Ensure audit tests do not inherit another migration test's schema state.""" @@ -36,28 +57,41 @@ def audit_database_env(postgres_database_url: str, migration_lock) -> str: async def audit_factory(audit_database_env: str): """Provide independent sessions over the isolated migrated database.""" engine = create_async_engine(audit_database_env) + async with engine.connect() as connection: + existing = set( + ( + await connection.execute( + text("select id from audit_events where event_domain = 'authority'") + ) + ).scalars() + ) try: yield async_sessionmaker(engine, expire_on_commit=False) finally: async with engine.begin() as connection: - await connection.execute( - text("lock table audit_events in access exclusive mode") - ) - await connection.execute( - text( - "alter table audit_events disable trigger " - "audit_events_reject_update_delete" + created = ( + set( + ( + await connection.execute( + text("select id from audit_events where event_domain = 'authority'") + ) + ).scalars() ) + - existing ) - await connection.execute( - text("delete from audit_events where event_domain = 'authority'") - ) - await connection.execute( - text( - "alter table audit_events enable trigger " - "audit_events_reject_update_delete" + if created: + await connection.execute(text("lock table audit_events in access exclusive mode")) + await connection.execute( + text( + "alter table audit_events disable trigger audit_events_reject_update_delete" + ) + ) + await connection.execute(delete(AuditEvent).where(AuditEvent.id.in_(created))) + await connection.execute( + text( + "alter table audit_events enable trigger audit_events_reject_update_delete" + ) ) - ) await engine.dispose() @@ -77,11 +111,27 @@ def _authority_input(event_type: AuthorityEventType, **overrides) -> AuthorityAu def test_authority_input_rejects_unbounded_or_inconsistent_evidence() -> None: - with pytest.raises(ValidationError, match="Extra inputs are not permitted"): - _authority_input( - AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED, - raw_token="must-never-enter-evidence", - ) + secret = "secret-bearer-value" + payload = _authority_input(AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED).model_dump( + mode="json" + ) | {"raw_token": secret} + constructors = ( + lambda: AuthorityAuditEventInput(**payload), + lambda: AuthorityAuditEventInput.model_validate(payload), + lambda: AuthorityAuditEventInput.model_validate_json(json.dumps(payload)), + ) + for constructor in constructors: + with pytest.raises(TypeError, match="unexpected authority audit fields") as caught: + constructor() + _assert_value_not_retained(caught.value, secret) + for patch in ( + {"actor_ref_kind": ActorReferenceKind.LEGACY_ACTOR, "actor_ref": "provider@example.com"}, + {"before_facts": {"status": "eyJ.raw.token"}}, + {"before_facts": {"decision_code": "https://issuer.example/private"}}, + {"reason": "See https://issuer.example/private"}, + ): + with pytest.raises(ValidationError): + _authority_input(AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED, **patch) with pytest.raises(ValidationError, match="reference fields must be paired"): _authority_input( AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED, @@ -110,6 +160,16 @@ def test_authority_input_rejects_unbounded_or_inconsistent_evidence() -> None: ) with pytest.raises(ValidationError, match="invalidation evidence"): _authority_input(AuthorityEventType.AUTHORITY_INVALIDATION_REQUESTED) + event_id = uuid4() + with pytest.raises(ValidationError, match="own event"): + _authority_input( + AuthorityEventType.AUTHORITY_INVALIDATION_REQUESTED, + event_id=event_id, + permission_id=None, + invalidation_cause_event_id=event_id, + invalidation_target_kind="actor", + invalidation_target_ref=str(uuid4()), + ) async def test_authority_writer_persists_typed_privacy_neutral_events(audit_factory) -> None: @@ -139,8 +199,7 @@ async def test_authority_writer_persists_typed_privacy_neutral_events(audit_fact async with audit_factory() as session: service = AuditService(session) stored = [ - await service.add_authority_event(value) - for value in (allowed, denied, invalidation) + await service.add_authority_event(value) for value in (allowed, denied, invalidation) ] await session.commit() @@ -203,6 +262,60 @@ async def test_authority_event_rollback_leaves_no_row(audit_factory) -> None: assert await session.get(AuditEvent, str(value.event_id)) is None +async def test_authority_invalidation_requires_an_existing_authority_cause(audit_factory) -> None: + legacy_id = str(uuid4()) + invalidation = _authority_input( + AuthorityEventType.AUTHORITY_INVALIDATION_REQUESTED, + permission_id=None, + invalidation_cause_event_id=uuid4(), + invalidation_target_kind="actor", + invalidation_target_ref=str(uuid4()), + ) + async with audit_factory() as session: + with pytest.raises(ValueError, match="existing authority event"): + await AuditService(session).add_authority_event(invalidation) + await AuditRepository(session).add_audit_event( + AuditEvent( + id=legacy_id, + entity_type="task", + entity_id=str(uuid4()), + event_type="task_created", + actor_id="legacy", + external_subject="opaque", + external_issuer="https://issuer.test", + actor_roles=[], + claim_snapshot={}, + auth_source="verified_token", + is_dev_auth=False, + event_payload={}, + ) + ) + invalidation.invalidation_cause_event_id = UUID(legacy_id) + with pytest.raises(ValueError, match="existing authority event"): + await AuditService(session).add_authority_event(invalidation) + assert await session.get(AuditEvent, str(invalidation.event_id)) is None + + +async def test_legacy_repository_rejects_unvalidated_authority_rows(audit_factory) -> None: + raw = AuditEvent( + id=str(uuid4()), + entity_type="actor", + entity_id=str(uuid4()), + event_type="SensitiveAuthorizationAllowed", + actor_id="provider@example.com", + actor_roles=[], + claim_snapshot={}, + auth_source="local_authority", + is_dev_auth=False, + event_payload={}, + event_domain="authority", + ) + async with audit_factory() as session: + with pytest.raises(ValueError, match="typed audit service"): + await AuditRepository(session).add_audit_event(raw) + assert raw not in session + + async def test_database_rejects_malformed_and_mutated_audit_rows(audit_factory) -> None: value = _authority_input(AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED) legacy_id = str(uuid4()) @@ -269,3 +382,58 @@ async def test_database_rejects_malformed_and_mutated_audit_rows(audit_factory) {"id": str(uuid4()), "entity_id": str(uuid4())}, ) await session.rollback() + insert = text( + "insert into audit_events " + "(id, entity_type, entity_id, event_type, actor_id, actor_roles, claim_snapshot, " + "auth_source, is_dev_auth, event_payload, event_domain, event_version, actor_ref_kind, " + "request_id, correlation_id, permission_id, before_facts) values " + "(:id, 'actor', :entity_id, 'SensitiveAuthorizationAllowed', :actor_id, '[]', '{}', " + "'local_authority', false, '{}', 'authority', 1, :actor_kind, :request_id, " + ":correlation_id, 'actor.manage', cast(:facts as json))" + ) + for actor_kind, actor_id, facts in ( + ("legacy_actor", "provider@example.com", "{}"), + ( + "system_principal", + "workstream:system:test", + '{"decision_code":"https://issuer.test"}', + ), + ("system_principal", "workstream:system:test", '{"status":"eyJ.raw.token"}'), + ): + with pytest.raises(IntegrityError): + await session.execute( + insert, + { + "id": str(uuid4()), + "entity_id": str(uuid4()), + "actor_id": actor_id, + "actor_kind": actor_kind, + "request_id": str(uuid4()), + "correlation_id": str(uuid4()), + "facts": facts, + }, + ) + await session.rollback() + invalidation_insert = text( + "insert into audit_events (id, entity_type, entity_id, event_type, actor_id, " + "actor_roles, claim_snapshot, auth_source, is_dev_auth, event_payload, event_domain, " + "event_version, actor_ref_kind, request_id, correlation_id, invalidation_cause_event_id, " + "invalidation_target_kind, invalidation_target_ref) values (:id, 'actor', :entity_id, " + "'AuthorityInvalidationRequested', 'workstream:system:test', '[]', '{}', " + "'local_authority', false, '{}', 'authority', 1, 'system_principal', :request_id, " + ":correlation_id, :cause_id, 'actor', :target_ref)" + ) + for cause_id in (str(uuid4()), legacy_id): + with pytest.raises(IntegrityError): + await session.execute( + invalidation_insert, + { + "id": str(uuid4()), + "entity_id": str(uuid4()), + "request_id": str(uuid4()), + "correlation_id": str(uuid4()), + "cause_id": cause_id, + "target_ref": str(uuid4()), + }, + ) + await session.rollback() From c93cf1442ca894251770f58ac1e22e98db7e1a3e Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 11:50:36 +0100 Subject: [PATCH 06/22] docs(auth): repair AUTH-05A privacy contract --- .agent-loop/LOOP_STATE.md | 12 ++-- .agent-loop/REVIEW_LOG.md | 17 +++++ .agent-loop/WORK_QUEUE.md | 2 +- .../CHUNK_MAP.md | 8 ++- .../STATUS.md | 15 ++-- ...001-05A-shared-audit-authority-evidence.md | 68 ++++++++++++++----- 6 files changed, 91 insertions(+), 31 deletions(-) diff --git a/.agent-loop/LOOP_STATE.md b/.agent-loop/LOOP_STATE.md index d18ddf6d..b6e97e29 100644 --- a/.agent-loop/LOOP_STATE.md +++ b/.agent-loop/LOOP_STATE.md @@ -10,14 +10,16 @@ - Worktree: `/home/abiorh/flow/workstream-auth-001-05` - Status: AUTH-04B post-merge memory merged through PR #114 as `97cd0f5`. The user explicitly started AUTH-05. Required L1 review rejected the combined - contract, then passed repaired AUTH-05A at `7a9023b`; bounded implementation - and deterministic evidence are active. + contract, then passed the first repaired AUTH-05A contract at `7a9023b`. + Exact-SHA implementation review reopened the contract for closed privacy + registries, non-echoing mapping admission, typed/SQL parity, and readable SQL. - Prior `WS-AUTH-001-01` reviewed implementation SHA: `be0b836` - Prior `WS-AUTH-001-01` final merged branch head: `b5217e1` - Latest integrated `main` merge commit: `97cd0f5` -- Current gate: AUTH-05A implementation evidence and exact-head internal review. -- Size checkpoint: each child has a 350-line inspection and a 500-line - production hard stop. +- Current gate: AUTH-05A repaired-contract internal review. +- Size checkpoint: AUTH-05A has a reviewer-required 500-line inspection and + 650-line production hard stop; security SQL may not be packed to satisfy it. + AUTH-05B retains its separate 350/500 boundary. - Next chunk: AUTH-05B remains inactive until AUTH-05A merges, post-merge memory is reconciled, and the user gives a separate explicit start. AUTH-06 remains inactive. diff --git a/.agent-loop/REVIEW_LOG.md b/.agent-loop/REVIEW_LOG.md index 397d4eed..6eaf047c 100644 --- a/.agent-loop/REVIEW_LOG.md +++ b/.agent-loop/REVIEW_LOG.md @@ -1,5 +1,22 @@ # Review Log +## 2026-07-14 - WS-AUTH-001-05A Contract Reopened By Exact-SHA Review + +Exact-SHA review of implementation repair `6fbb1f8` closed append-only, +repository ownership, canonical actor-reference, and invalidation-cause +integrity concerns, but found valid remaining privacy and auditability gaps. +Arbitrary reason/fact strings could still impersonate opaque secrets, rejected +known values and non-dict mappings remained recoverable through structured +Pydantic errors, direct SQL did not enforce every typed token bound, and the +500-line ceiling had encouraged unreadable security SQL. + +The repaired contract raises only AUTH-05A's ceiling to 650, prohibits +long-line packing, requires closed reason/fact registries, requires a +non-echoing pre-admission boundary for every Mapping, clarifies invalidation +cause integrity versus inactive idempotency replay, and requires typed/direct +SQL parity. No further runtime repair is permitted until required L1 contract +review passes; AUTH-05B remains inactive. + ## 2026-07-14 - WS-AUTH-001-05A Preimplementation Review Passed The repaired AUTH-05A contract passed required senior engineering, diff --git a/.agent-loop/WORK_QUEUE.md b/.agent-loop/WORK_QUEUE.md index e003c03d..2f6975d9 100644 --- a/.agent-loop/WORK_QUEUE.md +++ b/.agent-loop/WORK_QUEUE.md @@ -4,7 +4,7 @@ | Chunk | Title | Risk | Status | |---|---|---:|---| -| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | Repaired contract passed at `7a9023b`; bounded implementation and evidence active | +| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | Contract re-review for closed privacy registries and readable SQL | ## Planned Next diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md index 6ce3a9de..ac1caaaf 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md @@ -19,7 +19,7 @@ stopped. | `WS-AUTH-001-04A` | Request And Error Context | L1 | Merged through PR #111 as `90c9a28` | | `WS-AUTH-001-04B` | PostgreSQL Rate Controls | L1 | Merged through PR #113 as `05a63c8` | | `WS-AUTH-001-05` | Authority Evidence And Idempotency Foundation | L1 | Split before implementation into 05A and 05B | -| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | Implementation and deterministic evidence | +| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | Contract repair/re-review after exact-SHA privacy findings | | `WS-AUTH-001-05B` | Authority Idempotency And Invalidation Foundation | L1 | Inactive pending 05A merge/memory and separate explicit start | | `WS-AUTH-001-06` | Canonical Actor Profile And Identity Link | L1 | Proposed | | `WS-AUTH-001-07` | Authorization Kernel And Permission Registry | L1 | Proposed | @@ -97,5 +97,7 @@ merged as `05a63c8` after Backend, Agent Gates, CodeRabbit, and explicit human approval passed. AUTH-04B post-merge memory then merged through PR #114 as `97cd0f5`, and the user explicitly started AUTH-05. Required plan review rejected the combined contract before runtime edits and required 05A/05B. -Repaired contract review is the current gate. Do not implement either child -until its own review passes, and do not start AUTH-06 or POL-002-04. +The first 05A implementation review proved the original 500-line ceiling +incompatible with readable typed/database privacy parity. Repaired 05A contract +review is the current gate; 05B remains inactive. Do not start AUTH-06 or +POL-002-04. diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md index 7c7fbc6a..c3afde4f 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md @@ -46,8 +46,10 @@ explicit human approval merged PR #113 to `main` as `05a63c8` on 2026-07-14. AUTH-04B post-merge memory merged through PR #114 as `97cd0f5`. The user then explicitly started AUTH-05. Required L1 plan review rejected the combined audit/idempotency contract before runtime edits and required children 05A and -05B. The repaired AUTH-05A contract passed at `7a9023b`; its bounded runtime -implementation and deterministic evidence are active. +05B. The first repaired AUTH-05A contract passed at `7a9023b`, but exact-SHA +implementation review demonstrated that closed privacy registries and readable +typed/SQL parity require a repaired 650-line contract. Contract re-review is +active; no further runtime repair proceeds before it passes. ## Active planning chunk @@ -74,7 +76,7 @@ None. | `WS-AUTH-001-04A` | Merged | `codex/ws-auth-001-04-request-api-controls` | #111 | Merged as `90c9a28`; production review `cdcaf77`; final branch head `36c4aa5`. | | `WS-AUTH-001-04B` | Merged | `codex/ws-auth-001-04b-postgres-rate-controls` | #113 | Merged as `05a63c8`; production review `67484b5`; final branch head `94fb2fe`. | | `WS-AUTH-001-05` | Split | `codex/ws-auth-001-05-authority-evidence` | - | Parent split before implementation into 05A and 05B. | -| `WS-AUTH-001-05A` | Implementation/evidence | `codex/ws-auth-001-05-authority-evidence` | - | Repaired contract passed at `7a9023b`; shared audit ownership and append-only authority evidence active. | +| `WS-AUTH-001-05A` | Contract re-review | `codex/ws-auth-001-05-authority-evidence` | - | Closed privacy registries, non-echoing Mapping admission, typed/SQL parity, and readable SQL repair. | | `WS-AUTH-001-05B` | Inactive | - | - | Idempotency and invalidation; requires 05A merge/memory and separate explicit start. | | `WS-AUTH-001-06` | Proposed | - | - | Canonical actor profile and identity link. | | `WS-AUTH-001-07` | Proposed | - | - | Authorization kernel and permissions. | @@ -90,11 +92,14 @@ None. ## Blockers -AUTH-05A has no preimplementation blocker. The combined AUTH-05 contract was +AUTH-05A runtime repair is gated on its repaired-contract review. The combined AUTH-05 contract was rejected before code changes because migration `0017` is already owned by AUTH-04B and the shared-audit plus idempotency scope was not reviewable as one L1 change. Repaired AUTH-05A owns migration `0018`; AUTH-05B later owns -migration `0019`. Non-test +migration `0019`. Exact-SHA review found valid arbitrary fact/reason privacy, +non-dict Mapping retention, DB token-parity, cause-contract ambiguity, and SQL +auditability findings; the repaired contract owns their closure without +reactivating 05B. Non-test operators must later supply explicit classification evidence rather than inferred kinds before the owning canonical actor migration. diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md index 78984f01..fb5d8be7 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md @@ -2,8 +2,9 @@ ## Status -Active implementation and deterministic evidence. Repaired preimplementation -review passed on contract head `7a9023b`. +Contract repair and re-review after exact-SHA privacy findings. The initial +implementation review proved that safe typed/SQL parity did not fit the +original ceiling without compressing security-sensitive SQL. ## Parent initiative @@ -18,10 +19,15 @@ event and existing task/checker behavior. ## Risk and circuit breaker - Risk: L1 / SLA P1. -- Inspect scope at 350 changed non-comment production lines. -- Hard stop at 500 changed non-comment production lines, counting +- Inspect scope at 500 changed non-comment production lines. +- Hard stop at 650 changed non-comment production lines, counting `backend/app/**` plus migration code; tests and evidence do not justify exceeding the production limit. +- Security SQL must remain reviewable as named functions and multiline logical + clauses. Long-line packing does not reduce the circuit-breaker numerator and + is prohibited. +- This one-time ceiling repair is limited to the reviewer-required privacy and + causation controls; it does not reactivate AUTH-05B scope. ## Allowed files @@ -77,7 +83,7 @@ token from the adopted specification. Authority rows require: - optional target actor reference with all-or-none kind/reference presence; - optional matched-grant, permission, project, resource type/id, and target grant/link reference; -- optional bounded reason and denial code; +- optional closed reason code and denial code; - optional idempotency-record reference; - optional invalidation cause event and invalidation target kind/id; - typed shallow before/after fact objects. @@ -93,7 +99,7 @@ shapes using this exact compatibility matrix: actor-reference-kind field supplies its namespace; - existing `entity_type`/`entity_id` remain the primary audited aggregate reference; optional project/resource fields provide additional exact scope; -- existing `reason` may carry only the bounded authority reason; +- existing `reason` may carry only a registered authority reason code; - `from_status` and `to_status` are `NULL` because typed before/after facts own authority state transitions; - `external_subject` and `external_issuer` are `NULL`; @@ -106,11 +112,29 @@ Only the external subject/issuer columns become nullable for the conditional authority shape. Existing legacy rows and writers retain all current values and non-authority requirements. -Reference/type/reason tokens have explicit length and character bounds. -Before/after facts are JSON objects with at most 16 allowlisted scalar keys, -no nested containers, and at most 4096 encoded bytes per object. The typed -writer applies per-event allowed-key sets; database checks enforce object type -and byte bounds. +Reference/type tokens have explicit length and character bounds. Reason is not +free text: 05A registers only `automatic_first_access`, +`manual_service_provisioning`, `identity_link_change`, `actor_status_change`, +`authority_grant_change`, `qualification_decision`, +`permission_not_effective`, and `invalidation_requested`. + +Before/after facts are JSON objects with at most 16 allowlisted scalar keys, no +nested containers, and at most 4096 encoded bytes per object. Values are not +arbitrary bounded strings. `effective` is boolean; `scope_id` is a canonical +UUID; `subject_kind`, `provisioning_method`, `link_status`, `role_id`, +`scope_type`, `qualification_status`, `decision_code`, `status`, and +`target_status` use closed per-key registries derived from the adopted +specification. Typed validation and database constraints enforce the same key, +type, and value registries. Direct SQL cannot persist provider subjects, +emails, URLs, JWT-shaped values, or opaque token-like substitutes through +entity/type/reference, reason, or fact fields. + +The typed admission layer inspects every `collections.abc.Mapping`, including +non-dict mappings, before Pydantic owns rejected values. Unknown fields and +privacy-invalid known values raise one stable non-echoing error. Constructor, +`model_validate`, and `model_validate_json` tests traverse structured errors, +arguments, causes, contexts, and public object graphs to prove forbidden input +is not retained. AUTH-05A behavior tests exercise these exact foundation shapes: @@ -123,12 +147,18 @@ AUTH-05A behavior tests exercise these exact foundation shapes: target kind/reference, forbids denial code, and permits an optional canonical UUID idempotency reference for later AUTH-05B orchestration. -The 05A envelope and database constraints are final for this field: no foreign -key exists before AUTH-05B, and 05A introduces no lookup or replay behavior. -Tests persist both `NULL` foundation use and a syntactically valid non-NULL -future reference without creating an idempotency record. AUTH-05B consumes this -existing field without changing audit schema, constraints, repository, or -writer behavior. +The 05A envelope and database constraints are final for the idempotency-reference +field: it has no foreign key to an idempotency table before AUTH-05B, and 05A +introduces no idempotency lookup or replay behavior. Tests persist both `NULL` +foundation use and a syntactically valid non-NULL future reference without +creating an idempotency record. AUTH-05B consumes this existing field without +changing audit schema, constraints, repository, or writer behavior. + +Invalidation causation is a separate integrity boundary. Self, nonexistent, +and legacy-domain cause references are rejected; an invalidation cause must be +an existing authority event visible in the caller transaction. The typed +service and database insertion boundary enforce that rule without activating +an invalidation consumer or AUTH-05B replay behavior. Resource type/ID, target actor kind/reference, and invalidation target kind/reference are each all-or-none pairs in typed validation and database @@ -173,6 +203,10 @@ constraints. behaviorally unchanged. - Typed allowed, denied, and invalidation authority event shapes persist only bounded non-sensitive evidence and reject malformed/mixed shapes. +- Rejected provider/email/URL/token-like inputs are absent from every public + exception object graph for dict and non-dict mapping inputs. +- Typed and direct-SQL paths enforce closed reason/fact registries and matching + entity/type/reference privacy bounds. - Application-role normal DML cannot update, delete, or truncate any audit row. - Shared-writer tests instrument `AuditRepository` and prove TaskRepository compatibility methods pass the same session/event and return the shared From 2cd0fbedc437e6c181c698956846cbd3acaf5fd0 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 11:57:39 +0100 Subject: [PATCH 07/22] docs(auth): specify authority evidence registries --- .agent-loop/REVIEW_LOG.md | 11 ++ ...001-05A-shared-audit-authority-evidence.md | 151 ++++++++++++++---- 2 files changed, 133 insertions(+), 29 deletions(-) diff --git a/.agent-loop/REVIEW_LOG.md b/.agent-loop/REVIEW_LOG.md index 6eaf047c..7884c95e 100644 --- a/.agent-loop/REVIEW_LOG.md +++ b/.agent-loop/REVIEW_LOG.md @@ -1,5 +1,16 @@ # Review Log +## 2026-07-14 - WS-AUTH-001-05A Repaired Contract Review Failed + +Required review of contract `c93cf14` rejected implicit security registries. +Senior engineering, QA, and security agreed that exact event-to-reason, +event-to-fact, permission, denial, and reference matrices must be normative; +lexical bounds and "spec-derived" values were insufficient. Review also +required sanitized malformed-JSON handling, the two known checker regression +nodes, deterministic size/line checks, and symmetric additive test-delta +checks. The contract was repaired without runtime edits and requires a new +exact-SHA review. + ## 2026-07-14 - WS-AUTH-001-05A Contract Reopened By Exact-SHA Review Exact-SHA review of implementation repair `6fbb1f8` closed append-only, diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md index fb5d8be7..a41d4f79 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md @@ -79,7 +79,7 @@ token from the adopted specification. Authority rows require: - database-owned `occurred_at`; - `request_id` and `correlation_id` UUIDs; - namespaced acting reference kind (`legacy_actor`, `actor_profile`, or - `system_principal`) plus stable opaque reference; + `system_principal`) plus the exact local reference defined below; - optional target actor reference with all-or-none kind/reference presence; - optional matched-grant, permission, project, resource type/id, and target grant/link reference; @@ -112,36 +112,122 @@ Only the external subject/issuer columns become nullable for the conditional authority shape. Existing legacy rows and writers retain all current values and non-authority requirements. -Reference/type tokens have explicit length and character bounds. Reason is not -free text: 05A registers only `automatic_first_access`, -`manual_service_provisioning`, `identity_link_change`, `actor_status_change`, -`authority_grant_change`, `qualification_decision`, -`permission_not_effective`, and `invalidation_requested`. - -Before/after facts are JSON objects with at most 16 allowlisted scalar keys, no -nested containers, and at most 4096 encoded bytes per object. Values are not -arbitrary bounded strings. `effective` is boolean; `scope_id` is a canonical -UUID; `subject_kind`, `provisioning_method`, `link_status`, `role_id`, -`scope_type`, `qualification_status`, `decision_code`, `status`, and -`target_status` use closed per-key registries derived from the adopted -specification. Typed validation and database constraints enforce the same key, -type, and value registries. Direct SQL cannot persist provider subjects, -emails, URLs, JWT-shaped values, or opaque token-like substitutes through -entity/type/reference, reason, or fact fields. +Every authority string/reference field has one normative form: + +| Field | Allowed value | +|---|---| +| `entity_type` | `actor_profile`, `actor_identity_link`, `admin_role_grant`, `qualification_snapshot`, `project_role_grant`, `authorization_decision`, or `authority_invalidation` | +| `entity_id`, `matched_grant_id`, `project_id`, `resource_id`, `target_ref_id`, `invalidation_target_ref` | canonical lowercase UUID; nullable optional fields remain nullable | +| acting `legacy_actor` or `actor_profile` reference | canonical lowercase UUID | +| acting `system_principal` reference | `workstream:system:bootstrap`; additional principals require an approved registry change | +| target actor | kind `actor_profile` plus canonical lowercase UUID | +| `resource_type` | `actor_profile`, `actor_identity_link`, `admin_role_grant`, `project`, `project_role_grant`, `task`, `submission`, `review`, `contribution`, `compensation_award`, `compensation_delivery`, `operations`, or `audit_event` | +| target/invalidation kind | `actor_profile`, `actor_identity_link`, `admin_role_grant`, `qualification_snapshot`, `project_role_grant`, `authorization_decision`, or `permission_registry` | +| UUID envelope fields | native/canonical UUID for event, request, correlation, idempotency, and cause references | + +`permission_id` is closed to the exact specification Section 11 catalog: +`actor.profile.read_self`, `actor.profile.update_self`, +`actor.profile.read_any`, `actor.profile.suspend`, `actor.profile.reactivate`, +`actor.profile.deactivate`, `actor.identity_link.read`, +`actor.identity_link.revoke`, `actor.identity_link.reactivate`, +`actor.service.provision`, `admin_role.read`, `admin_role.grant`, +`admin_role.revoke`, `project.create`, `project.read`, `project.update`, +`project.archive`, `project.guide.manage`, `project.effective_policy.manage`, +`project.task.manage`, `project.review_policy.manage`, +`project.role_grant.read`, `project.role_grant.manage`, `task.queue.read`, +`task.claim`, `submission.create`, `submission.read_own`, +`submission.read_for_review`, `review.queue.read`, `review.queue.inspect`, +`review.claim`, `review.release`, `review.decline_preference`, +`review.decision`, `review.lease.force_release`, `review.chain.read`, +`contribution.read_self`, `contribution.read_project`, +`compensation.policy.manage`, `compensation.adapter_binding.manage`, +`compensation.award.read`, `compensation.delivery.reconcile`, +`operations.status.read`, `operations.timer.run`, +`operations.reconcile.run`, `operations.outbox.retry`, +`operations.projection.rebuild`, `audit.read`, and `audit.export`. + +`denial_code` is closed to the exact specification Section 22 codes. The +authority writer may use only: `required_scope_missing`, +`unsupported_subject_kind`, `service_actor_not_provisioned`, +`identity_link_revoked`, `actor_suspended`, `actor_deactivated`, +`permission_not_granted`, `scope_not_authorized`, `self_grant_forbidden`, +`self_role_revoke_forbidden`, `resource_guard_denied`, `actor_not_found`, +`grant_not_found`, `resource_not_found`, `actor_already_suspended`, +`actor_not_suspended`, `actor_deactivated_terminal`, +`last_access_administrator`, `admin_role_grant_exists`, +`project_role_grant_exists`, `identity_link_conflict`, +`resource_project_mismatch`, `idempotency_mismatch`, `invalid_role_scope`, +`invalid_project_role`, or `qualification_snapshot_invalid`. + +Audit `reason` is a privacy-safe classification, not operator-entered text and +not a replacement for the required rationale stored on the owning domain row. +The writer never copies domain reason text into audit evidence. Reason is +required and event-compatible as follows: + +| Event group | Allowed reason classification | +|---|---| +| human/service provisioning | `automatic_first_access` / `manual_service_provisioning`, respectively | +| identity link create/revoke/reactivate | `identity_lifecycle_change` | +| actor suspend/deactivate | `security_response` or `administrative_correction` | +| actor reactivate | `administrative_correction` | +| initial Access Administrator bootstrap | `initial_access_bootstrap` | +| admin/project grant issue | `authority_assignment` | +| project grant replacement | `authority_replacement` | +| admin/project grant revoke | `authority_revocation` | +| admin grant/last-admin denial | `authorization_policy_denial` | +| qualification snapshot | `qualification_evidence_captured` | +| sensitive allow/deny | `authorization_evaluation` | +| invalidation request | `authority_state_changed` | + +Before/after facts are JSON objects with at most seven allowlisted scalar keys, +no nested containers, and at most 4096 encoded bytes. The only fact keys and +values are: `status` in `active`, `suspended`, `deactivated`, `revoked`, or +`captured`; `subject_kind` in `human` or `service`; `provisioning_method` in +`automatic_first_access` or `manual_service_provisioning`; `role` in +`access_administrator`, `operator`, `project_manager`, `finance_authority`, +`audit_authority`, `submitter`, `reviewer`, or `both`; `scope_type` in `system` +or `project`; `scope_id` as a canonical UUID; and `effective` as boolean. + +The event-specific fact matrix is exact: + +| Event group | Before facts | After facts | +|---|---|---| +| human/service provisioning | `NULL` | `status=active`, matching `subject_kind` and `provisioning_method` | +| identity linked | `NULL` | `status=active`, `subject_kind` | +| identity revoked/reactivated | `status=active` / `status=revoked` | `status=revoked` / `status=active` | +| actor suspended/reactivated | `status=active` / `status=suspended` | `status=suspended` / `status=active` | +| actor deactivated | `status=active` or `suspended` | `status=deactivated` | +| initial admin bootstrap | `NULL` | `status=active`, `role=access_administrator`, `scope_type=system`, `effective=true` | +| admin/project grant issued | `NULL` | `status=active`, applicable `role`/scope, `effective=true` | +| project grant replaced | prior active role/scope with `effective=true` | replacement active role/scope with `effective=true` | +| admin/project grant revoked | active role/scope with `effective=true` | same role/scope, `status=revoked`, `effective=false` | +| qualification snapshot captured | `NULL` | `status=captured` | +| grant operation denied or last-admin denial | `NULL` | `effective=false` | +| sensitive allowed/denied | `NULL` | `effective=true` / `effective=false` | +| invalidation requested | `effective=true` | `effective=false` | + +For grant facts, admin roles use `system` or `project`; project roles use +`project`; `scope_id` is absent for system scope and required for project scope. +Typed validation and database constraints enforce the identical envelope and +fact matrices. Table-driven tests execute the same positive and negative cases +through Pydantic and direct SQL; lexical bounds alone are insufficient. The typed admission layer inspects every `collections.abc.Mapping`, including non-dict mappings, before Pydantic owns rejected values. Unknown fields and -privacy-invalid known values raise one stable non-echoing error. Constructor, -`model_validate`, and `model_validate_json` tests traverse structured errors, -arguments, causes, contexts, and public object graphs to prove forbidden input -is not retained. +privacy-invalid known values raise one stable non-echoing error. A sanitized +`AuthorityAuditEventInput.model_validate_json` override replaces malformed, +scalar, list, string, bytes, and bytearray decode/type failures with that same +value-free error instead of exposing a decoder `.doc` or Pydantic input. +Constructor, `model_validate`, and `model_validate_json` tests traverse +structured errors, arguments, causes, contexts, and public object graphs to +prove forbidden input is not retained. AUTH-05A behavior tests exercise these exact foundation shapes: - `SensitiveAuthorizationAllowed` requires permission, forbids denial code and invalidation cause/target, and permits an optional canonical UUID idempotency reference for later AUTH-05B orchestration; -- `SensitiveAuthorizationDenied` requires permission plus a bounded denial +- `SensitiveAuthorizationDenied` requires permission plus a registered denial code and forbids invalidation cause/target and idempotency reference; - `AuthorityInvalidationRequested` requires cause-event ID plus all-or-none target kind/reference, forbids denial code, and permits an optional canonical @@ -224,14 +310,13 @@ trap 'rm -rf "$tmp_dir"' EXIT .venv/bin/python -m pytest -q \ tests/test_alembic.py::test_authority_audit_schema_preserves_legacy_and_guards_downgrade \ tests/test_audit.py \ - tests/test_tasks.py::test_task_repository_delegates_audit_persistence + tests/test_tasks.py::test_task_repository_delegates_audit_persistence \ + tests/test_tasks.py::test_manual_checker_run_cannot_bypass_failed_automatic_gate \ + tests/test_tasks.py::test_queued_gate_fails_closed_when_lock_audit_is_missing .venv/bin/python scripts/run_isolated_tests.py --metadata-json "$tmp_dir/05a-coverage.json" --timeout-seconds 1800 -- \ .venv/bin/python -m pytest -q tests/test_audit.py \ tests/test_tasks.py::test_task_repository_delegates_audit_persistence \ --cov=app.modules.audit --cov-report=term-missing --cov-fail-under=90 -.venv/bin/python scripts/run_isolated_tests.py --metadata-json "$tmp_dir/full.json" --timeout-seconds 12600 -- \ - .venv/bin/python -m pytest -q --ignore=tests/test_isolated_database_runner.py \ - --cov=app --cov-report=term-missing --cov-fail-under=78 .venv/bin/ruff check app tests .venv/bin/docstr-coverage --config .docstr.yaml cd .. @@ -240,14 +325,22 @@ python3 scripts/check_stale_authorization_docs.py python3 scripts/check_stale_artifact_contracts.py python3 scripts/check_markdown_links.py python3 scripts/check_internal_review_evidence.py +test "$(git diff --unified=0 origin/main...HEAD -- backend/app backend/alembic | \ + awk '/^\+\+\+|^---/{next} /^\+/{line=substr($0,2); if (line !~ /^[[:space:]]*($|#)/) n++} END{print n+0}')" -le 650 +awk 'length($0) > 120 { print FNR ":" $0; bad=1 } END { exit bad }' \ + backend/alembic/versions/0018_authority_audit_evidence.py +if git diff --unified=0 origin/main...HEAD -- backend/tests | \ + rg '^-(.*assert|.*pytest\.raises|.*pytest\.mark\.(skip|xfail)|.*pytest\.(skip|xfail)|.*skipTest)'; then exit 1; fi if git diff --unified=0 origin/main...HEAD -- backend/tests | \ - rg '^-(.*assert|.*pytest\.raises|.*pytest\.mark\.(skip|xfail)|.*skipTest)'; then exit 1; fi + rg '^\+(.*pytest\.mark\.(skip|xfail)|.*pytest\.(skip|xfail)|.*skipTest)'; then exit 1; fi git diff --check ``` Run the exact migration node before audit/delegation tests on the same isolated -database. Test delta must be additive: no assertion, raises, skip, xfail, -threshold, workflow, or exclusion weakening. +database. Do not repeat the multi-hour local full suite after focused repairs; +GitHub Backend CI remains the final repository-wide `--cov-fail-under=78` gate. +Test delta must be additive: no assertion, raises, skip, xfail, threshold, +workflow, or exclusion weakening. ## Required reviewers From 27d9ecaa42aceb5110f00b2d1828f84f00f7b8b9 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 12:02:59 +0100 Subject: [PATCH 08/22] docs(auth): resolve authority registry contradictions --- .agent-loop/REVIEW_LOG.md | 6 +++ ...001-05A-shared-audit-authority-evidence.md | 53 +++++++++++++------ 2 files changed, 43 insertions(+), 16 deletions(-) diff --git a/.agent-loop/REVIEW_LOG.md b/.agent-loop/REVIEW_LOG.md index 7884c95e..7af00934 100644 --- a/.agent-loop/REVIEW_LOG.md +++ b/.agent-loop/REVIEW_LOG.md @@ -11,6 +11,12 @@ nodes, deterministic size/line checks, and symmetric additive test-delta checks. The contract was repaired without runtime edits and requires a new exact-SHA review. +Review of `2cd0fbe` then caught contradictory UUID rules for permission targets, +resource-ID nullability, admin role/scope compatibility, optional-versus-required +reason wording, and state-versus-decision fact meaning. It also required exact +added-plus-deleted size arithmetic and reuse of the repository's AST-aware test +weakening detector. Those contract defects were repaired before runtime work. + ## 2026-07-14 - WS-AUTH-001-05A Contract Reopened By Exact-SHA Review Exact-SHA review of implementation repair `6fbb1f8` closed append-only, diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md index a41d4f79..835c2c9d 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md @@ -83,7 +83,7 @@ token from the adopted specification. Authority rows require: - optional target actor reference with all-or-none kind/reference presence; - optional matched-grant, permission, project, resource type/id, and target grant/link reference; -- optional closed reason code and denial code; +- required authority reason classification and optional denial code; - optional idempotency-record reference; - optional invalidation cause event and invalidation target kind/id; - typed shallow before/after fact objects. @@ -117,12 +117,12 @@ Every authority string/reference field has one normative form: | Field | Allowed value | |---|---| | `entity_type` | `actor_profile`, `actor_identity_link`, `admin_role_grant`, `qualification_snapshot`, `project_role_grant`, `authorization_decision`, or `authority_invalidation` | -| `entity_id`, `matched_grant_id`, `project_id`, `resource_id`, `target_ref_id`, `invalidation_target_ref` | canonical lowercase UUID; nullable optional fields remain nullable | +| `entity_id`, `matched_grant_id`, `project_id`, `resource_id` | canonical lowercase UUID; nullable optional fields remain nullable; decision/invalidation `entity_id` equals its event ID | | acting `legacy_actor` or `actor_profile` reference | canonical lowercase UUID | | acting `system_principal` reference | `workstream:system:bootstrap`; additional principals require an approved registry change | | target actor | kind `actor_profile` plus canonical lowercase UUID | | `resource_type` | `actor_profile`, `actor_identity_link`, `admin_role_grant`, `project`, `project_role_grant`, `task`, `submission`, `review`, `contribution`, `compensation_award`, `compensation_delivery`, `operations`, or `audit_event` | -| target/invalidation kind | `actor_profile`, `actor_identity_link`, `admin_role_grant`, `qualification_snapshot`, `project_role_grant`, `authorization_decision`, or `permission_registry` | +| target/invalidation kind and reference | `actor_profile`, `actor_identity_link`, `admin_role_grant`, `qualification_snapshot`, or `project_role_grant` uses a canonical UUID; `permission_registry` uses one registered `permission_id` below | | UUID envelope fields | native/canonical UUID for event, request, correlation, idempotency, and cause references | `permission_id` is closed to the exact specification Section 11 catalog: @@ -179,14 +179,16 @@ required and event-compatible as follows: | sensitive allow/deny | `authorization_evaluation` | | invalidation request | `authority_state_changed` | -Before/after facts are JSON objects with at most seven allowlisted scalar keys, +Before/after facts are JSON objects with at most eight allowlisted scalar keys, no nested containers, and at most 4096 encoded bytes. The only fact keys and values are: `status` in `active`, `suspended`, `deactivated`, `revoked`, or `captured`; `subject_kind` in `human` or `service`; `provisioning_method` in `automatic_first_access` or `manual_service_provisioning`; `role` in `access_administrator`, `operator`, `project_manager`, `finance_authority`, `audit_authority`, `submitter`, `reviewer`, or `both`; `scope_type` in `system` -or `project`; `scope_id` as a canonical UUID; and `effective` as boolean. +or `project`; `scope_id` as a canonical UUID; and `effective` and `allowed` as +booleans. `effective` describes persisted authority state; `allowed` describes +an authorization decision and they are not interchangeable. The event-specific fact matrix is exact: @@ -202,12 +204,14 @@ The event-specific fact matrix is exact: | project grant replaced | prior active role/scope with `effective=true` | replacement active role/scope with `effective=true` | | admin/project grant revoked | active role/scope with `effective=true` | same role/scope, `status=revoked`, `effective=false` | | qualification snapshot captured | `NULL` | `status=captured` | -| grant operation denied or last-admin denial | `NULL` | `effective=false` | -| sensitive allowed/denied | `NULL` | `effective=true` / `effective=false` | +| grant operation denied or last-admin denial | `NULL` | `NULL`; `denial_code` carries the attempted-operation result | +| sensitive allowed/denied | `NULL` | `allowed=true` / `allowed=false` | | invalidation requested | `effective=true` | `effective=false` | -For grant facts, admin roles use `system` or `project`; project roles use -`project`; `scope_id` is absent for system scope and required for project scope. +For grant facts, `access_administrator` and `operator` use only `system`; +`project_manager`, `finance_authority`, and `audit_authority` use `system` or +`project`; `submitter`, `reviewer`, and `both` use only `project`. `scope_id` is +absent for system scope and required for project scope. Typed validation and database constraints enforce the identical envelope and fact matrices. Table-driven tests execute the same positive and negative cases through Pydantic and direct SQL; lexical bounds alone are insufficient. @@ -246,9 +250,10 @@ an existing authority event visible in the caller transaction. The typed service and database insertion boundary enforce that rule without activating an invalidation consumer or AUTH-05B replay behavior. -Resource type/ID, target actor kind/reference, and invalidation target -kind/reference are each all-or-none pairs in typed validation and database -constraints. +Target actor kind/reference and target/invalidation kind/reference are +all-or-none pairs. `resource_id` requires `resource_type`, but `resource_type` +may have a `NULL` ID for a system-wide decision as allowed by the adopted +`AuthorizationDecision` contract. ## Shared writer and append-only custody @@ -325,14 +330,30 @@ python3 scripts/check_stale_authorization_docs.py python3 scripts/check_stale_artifact_contracts.py python3 scripts/check_markdown_links.py python3 scripts/check_internal_review_evidence.py -test "$(git diff --unified=0 origin/main...HEAD -- backend/app backend/alembic | \ - awk '/^\+\+\+|^---/{next} /^\+/{line=substr($0,2); if (line !~ /^[[:space:]]*($|#)/) n++} END{print n+0}')" -le 650 +changed_production_lines=$(git diff --unified=0 origin/main...HEAD -- backend/app backend/alembic | \ + awk '/^\+\+\+|^---/{next} /^[+-]/{line=substr($0,2); if (line !~ /^[[:space:]]*($|#)/) n++} END{print n+0}') +printf 'AUTH-05A changed production lines: %s\n' "$changed_production_lines" +if test "$changed_production_lines" -gt 500; then printf 'AUTH-05A inspection required\n'; fi +test "$changed_production_lines" -le 650 awk 'length($0) > 120 { print FNR ":" $0; bad=1 } END { exit bad }' \ backend/alembic/versions/0018_authority_audit_evidence.py +.venv/bin/python - <<'PY' +from pathlib import Path +import subprocess +import sys + +sys.path.insert(0, "scripts") +from coverage_policy import weak_python # noqa: E402 + +paths = subprocess.check_output( + ["git", "diff", "--relative", "--name-only", "origin/main...HEAD", "--", "tests"], + text=True, +).splitlines() +blocked = [path for path in paths if path.endswith(".py") and weak_python(Path(path))] +raise SystemExit(f"test weakening: {blocked}" if blocked else 0) +PY if git diff --unified=0 origin/main...HEAD -- backend/tests | \ rg '^-(.*assert|.*pytest\.raises|.*pytest\.mark\.(skip|xfail)|.*pytest\.(skip|xfail)|.*skipTest)'; then exit 1; fi -if git diff --unified=0 origin/main...HEAD -- backend/tests | \ - rg '^\+(.*pytest\.mark\.(skip|xfail)|.*pytest\.(skip|xfail)|.*skipTest)'; then exit 1; fi git diff --check ``` From 7cc6058eae902fbbc3478de0a72d8d214fbb9b3b Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 12:06:12 +0100 Subject: [PATCH 09/22] docs(auth): bind authority evidence scope --- .agent-loop/REVIEW_LOG.md | 6 ++++ ...001-05A-shared-audit-authority-evidence.md | 33 +++++++++++-------- 2 files changed, 25 insertions(+), 14 deletions(-) diff --git a/.agent-loop/REVIEW_LOG.md b/.agent-loop/REVIEW_LOG.md index 7af00934..59a89122 100644 --- a/.agent-loop/REVIEW_LOG.md +++ b/.agent-loop/REVIEW_LOG.md @@ -16,6 +16,12 @@ resource-ID nullability, admin role/scope compatibility, optional-versus-require reason wording, and state-versus-decision fact meaning. It also required exact added-plus-deleted size arithmetic and reuse of the repository's AST-aware test weakening detector. Those contract defects were repaired before runtime work. +QA then found the AST command below the repository-root directory change; the +command was moved into its executable backend-relative position before final +contract review. +Senior review also required project-scoped grant facts to match the envelope +project, replacement facts to retain one scope, system-scope grant evidence to +omit project scope, and project resources to agree with the envelope project. ## 2026-07-14 - WS-AUTH-001-05A Contract Reopened By Exact-SHA Review diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md index 835c2c9d..c9048bf3 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md @@ -211,7 +211,12 @@ The event-specific fact matrix is exact: For grant facts, `access_administrator` and `operator` use only `system`; `project_manager`, `finance_authority`, and `audit_authority` use `system` or `project`; `submitter`, `reviewer`, and `both` use only `project`. `scope_id` is -absent for system scope and required for project scope. +absent for system scope and required for project scope. System-scope grant +evidence has `project_id = NULL`. Project-scope grant evidence requires +`project_id`, and every before/after `scope_id` equals that envelope value; +replacement retains the same scope ID before and after. When +`resource_type = project` and `resource_id` is present, `resource_id` also +equals `project_id`. Typed and direct-SQL paths enforce these cross-field rules. Typed validation and database constraints enforce the identical envelope and fact matrices. Table-driven tests execute the same positive and negative cases through Pydantic and direct SQL; lexical bounds alone are insufficient. @@ -324,19 +329,6 @@ trap 'rm -rf "$tmp_dir"' EXIT --cov=app.modules.audit --cov-report=term-missing --cov-fail-under=90 .venv/bin/ruff check app tests .venv/bin/docstr-coverage --config .docstr.yaml -cd .. -python3 scripts/check_stale_workstream_wording.py -python3 scripts/check_stale_authorization_docs.py -python3 scripts/check_stale_artifact_contracts.py -python3 scripts/check_markdown_links.py -python3 scripts/check_internal_review_evidence.py -changed_production_lines=$(git diff --unified=0 origin/main...HEAD -- backend/app backend/alembic | \ - awk '/^\+\+\+|^---/{next} /^[+-]/{line=substr($0,2); if (line !~ /^[[:space:]]*($|#)/) n++} END{print n+0}') -printf 'AUTH-05A changed production lines: %s\n' "$changed_production_lines" -if test "$changed_production_lines" -gt 500; then printf 'AUTH-05A inspection required\n'; fi -test "$changed_production_lines" -le 650 -awk 'length($0) > 120 { print FNR ":" $0; bad=1 } END { exit bad }' \ - backend/alembic/versions/0018_authority_audit_evidence.py .venv/bin/python - <<'PY' from pathlib import Path import subprocess @@ -352,6 +344,19 @@ paths = subprocess.check_output( blocked = [path for path in paths if path.endswith(".py") and weak_python(Path(path))] raise SystemExit(f"test weakening: {blocked}" if blocked else 0) PY +cd .. +python3 scripts/check_stale_workstream_wording.py +python3 scripts/check_stale_authorization_docs.py +python3 scripts/check_stale_artifact_contracts.py +python3 scripts/check_markdown_links.py +python3 scripts/check_internal_review_evidence.py +changed_production_lines=$(git diff --unified=0 origin/main...HEAD -- backend/app backend/alembic | \ + awk '/^\+\+\+|^---/{next} /^[+-]/{line=substr($0,2); if (line !~ /^[[:space:]]*($|#)/) n++} END{print n+0}') +printf 'AUTH-05A changed production lines: %s\n' "$changed_production_lines" +if test "$changed_production_lines" -gt 500; then printf 'AUTH-05A inspection required\n'; fi +test "$changed_production_lines" -le 650 +awk 'length($0) > 120 { print FNR ":" $0; bad=1 } END { exit bad }' \ + backend/alembic/versions/0018_authority_audit_evidence.py if git diff --unified=0 origin/main...HEAD -- backend/tests | \ rg '^-(.*assert|.*pytest\.raises|.*pytest\.mark\.(skip|xfail)|.*pytest\.(skip|xfail)|.*skipTest)'; then exit 1; fi git diff --check From 42c0a4df3c41d860c2f75b9b523c1e927f98fe9b Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 12:08:20 +0100 Subject: [PATCH 10/22] docs(auth): record AUTH-05A contract approval --- .agent-loop/LOOP_STATE.md | 3 ++- .agent-loop/REVIEW_LOG.md | 9 +++++++++ .agent-loop/WORK_QUEUE.md | 2 +- .../CHUNK_MAP.md | 6 +++--- .../STATUS.md | 7 ++++--- .../WS-AUTH-001-05A-shared-audit-authority-evidence.md | 7 ++++--- 6 files changed, 23 insertions(+), 11 deletions(-) diff --git a/.agent-loop/LOOP_STATE.md b/.agent-loop/LOOP_STATE.md index b6e97e29..d60b1d79 100644 --- a/.agent-loop/LOOP_STATE.md +++ b/.agent-loop/LOOP_STATE.md @@ -16,7 +16,8 @@ - Prior `WS-AUTH-001-01` reviewed implementation SHA: `be0b836` - Prior `WS-AUTH-001-01` final merged branch head: `b5217e1` - Latest integrated `main` merge commit: `97cd0f5` -- Current gate: AUTH-05A repaired-contract internal review. +- Current gate: AUTH-05A bounded runtime repair and focused evidence after the + repaired contract passed exact-SHA review at `7cc6058`. - Size checkpoint: AUTH-05A has a reviewer-required 500-line inspection and 650-line production hard stop; security SQL may not be packed to satisfy it. AUTH-05B retains its separate 350/500 boundary. diff --git a/.agent-loop/REVIEW_LOG.md b/.agent-loop/REVIEW_LOG.md index 59a89122..7bfd737b 100644 --- a/.agent-loop/REVIEW_LOG.md +++ b/.agent-loop/REVIEW_LOG.md @@ -1,5 +1,14 @@ # Review Log +## 2026-07-14 - WS-AUTH-001-05A Repaired Contract Review Passed + +The exact contract at `7cc6058` passed senior engineering, architecture, +product/ops, docs, reuse, QA, CI/test-delta, and security/privacy review. It +locks closed envelope/reason/fact registries, cross-field project integrity, +non-echoing Mapping/JSON admission, identical typed/direct-SQL behavior, +readable SQL, focused regressions, and the additions-plus-deletions 500/650 +circuit breaker. Bounded runtime repair may proceed; AUTH-05B remains inactive. + ## 2026-07-14 - WS-AUTH-001-05A Repaired Contract Review Failed Required review of contract `c93cf14` rejected implicit security registries. diff --git a/.agent-loop/WORK_QUEUE.md b/.agent-loop/WORK_QUEUE.md index 2f6975d9..2b74e70b 100644 --- a/.agent-loop/WORK_QUEUE.md +++ b/.agent-loop/WORK_QUEUE.md @@ -4,7 +4,7 @@ | Chunk | Title | Risk | Status | |---|---|---:|---| -| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | Contract re-review for closed privacy registries and readable SQL | +| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | Repaired contract passed at `7cc6058`; bounded runtime repair and focused evidence | ## Planned Next diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md index ac1caaaf..6c9dc4c4 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md @@ -19,7 +19,7 @@ stopped. | `WS-AUTH-001-04A` | Request And Error Context | L1 | Merged through PR #111 as `90c9a28` | | `WS-AUTH-001-04B` | PostgreSQL Rate Controls | L1 | Merged through PR #113 as `05a63c8` | | `WS-AUTH-001-05` | Authority Evidence And Idempotency Foundation | L1 | Split before implementation into 05A and 05B | -| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | Contract repair/re-review after exact-SHA privacy findings | +| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | Repaired contract passed at `7cc6058`; runtime repair/evidence | | `WS-AUTH-001-05B` | Authority Idempotency And Invalidation Foundation | L1 | Inactive pending 05A merge/memory and separate explicit start | | `WS-AUTH-001-06` | Canonical Actor Profile And Identity Link | L1 | Proposed | | `WS-AUTH-001-07` | Authorization Kernel And Permission Registry | L1 | Proposed | @@ -99,5 +99,5 @@ approval passed. AUTH-04B post-merge memory then merged through PR #114 as rejected the combined contract before runtime edits and required 05A/05B. The first 05A implementation review proved the original 500-line ceiling incompatible with readable typed/database privacy parity. Repaired 05A contract -review is the current gate; 05B remains inactive. Do not start AUTH-06 or -POL-002-04. +review passed at `7cc6058`; bounded runtime repair is active. 05B remains +inactive. Do not start AUTH-06 or POL-002-04. diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md index c3afde4f..23fa23e9 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md @@ -49,7 +49,7 @@ audit/idempotency contract before runtime edits and required children 05A and 05B. The first repaired AUTH-05A contract passed at `7a9023b`, but exact-SHA implementation review demonstrated that closed privacy registries and readable typed/SQL parity require a repaired 650-line contract. Contract re-review is -active; no further runtime repair proceeds before it passes. +complete at `7cc6058`; bounded runtime repair and focused evidence are active. ## Active planning chunk @@ -76,7 +76,7 @@ None. | `WS-AUTH-001-04A` | Merged | `codex/ws-auth-001-04-request-api-controls` | #111 | Merged as `90c9a28`; production review `cdcaf77`; final branch head `36c4aa5`. | | `WS-AUTH-001-04B` | Merged | `codex/ws-auth-001-04b-postgres-rate-controls` | #113 | Merged as `05a63c8`; production review `67484b5`; final branch head `94fb2fe`. | | `WS-AUTH-001-05` | Split | `codex/ws-auth-001-05-authority-evidence` | - | Parent split before implementation into 05A and 05B. | -| `WS-AUTH-001-05A` | Contract re-review | `codex/ws-auth-001-05-authority-evidence` | - | Closed privacy registries, non-echoing Mapping admission, typed/SQL parity, and readable SQL repair. | +| `WS-AUTH-001-05A` | Runtime repair/evidence | `codex/ws-auth-001-05-authority-evidence` | - | Repaired contract passed at `7cc6058`; closed registries, non-echoing admission, typed/SQL parity, and readable SQL. | | `WS-AUTH-001-05B` | Inactive | - | - | Idempotency and invalidation; requires 05A merge/memory and separate explicit start. | | `WS-AUTH-001-06` | Proposed | - | - | Canonical actor profile and identity link. | | `WS-AUTH-001-07` | Proposed | - | - | Authorization kernel and permissions. | @@ -92,7 +92,8 @@ None. ## Blockers -AUTH-05A runtime repair is gated on its repaired-contract review. The combined AUTH-05 contract was +AUTH-05A has no preimplementation blocker after repaired-contract review passed +at `7cc6058`. The combined AUTH-05 contract was rejected before code changes because migration `0017` is already owned by AUTH-04B and the shared-audit plus idempotency scope was not reviewable as one L1 change. Repaired AUTH-05A owns migration `0018`; AUTH-05B later owns diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md index c9048bf3..b15b4c8c 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md @@ -2,9 +2,10 @@ ## Status -Contract repair and re-review after exact-SHA privacy findings. The initial -implementation review proved that safe typed/SQL parity did not fit the -original ceiling without compressing security-sensitive SQL. +Bounded runtime repair and focused evidence. Repaired contract review passed on +exact SHA `7cc6058` after the initial implementation review proved that safe +typed/SQL parity did not fit the original ceiling without compressing +security-sensitive SQL. ## Parent initiative From ab62a617b6b06048be1fd28581954f2c1fe5d89f Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 12:16:50 +0100 Subject: [PATCH 11/22] docs(auth): raise AUTH-05A circuit breaker --- .agent-loop/LOOP_STATE.md | 7 ++++--- .agent-loop/REVIEW_LOG.md | 13 ++++++++++++- .agent-loop/WORK_QUEUE.md | 2 +- .../CHUNK_MAP.md | 2 +- .../STATUS.md | 7 +++++-- ...UTH-001-05A-shared-audit-authority-evidence.md | 15 +++++++++------ 6 files changed, 32 insertions(+), 14 deletions(-) diff --git a/.agent-loop/LOOP_STATE.md b/.agent-loop/LOOP_STATE.md index d60b1d79..433aea5c 100644 --- a/.agent-loop/LOOP_STATE.md +++ b/.agent-loop/LOOP_STATE.md @@ -16,10 +16,11 @@ - Prior `WS-AUTH-001-01` reviewed implementation SHA: `be0b836` - Prior `WS-AUTH-001-01` final merged branch head: `b5217e1` - Latest integrated `main` merge commit: `97cd0f5` -- Current gate: AUTH-05A bounded runtime repair and focused evidence after the - repaired contract passed exact-SHA review at `7cc6058`. +- Current gate: AUTH-05A 1000-line circuit-breaker amendment review after the + 650-line runtime dry run stopped without committing production edits. - Size checkpoint: AUTH-05A has a reviewer-required 500-line inspection and - 650-line production hard stop; security SQL may not be packed to satisfy it. + 1000-line production hard stop after the closed typed/SQL matrix dry run; + security SQL may not be packed to satisfy it. AUTH-05B retains its separate 350/500 boundary. - Next chunk: AUTH-05B remains inactive until AUTH-05A merges, post-merge memory is reconciled, and the user gives a separate explicit start. AUTH-06 remains diff --git a/.agent-loop/REVIEW_LOG.md b/.agent-loop/REVIEW_LOG.md index 7bfd737b..8d797b4e 100644 --- a/.agent-loop/REVIEW_LOG.md +++ b/.agent-loop/REVIEW_LOG.md @@ -1,12 +1,23 @@ # Review Log +## 2026-07-14 - WS-AUTH-001-05A Circuit Breaker Reopened + +The first post-approval exact-registry dry run reached 689 changed non-comment +production lines before readable PostgreSQL parity was complete, crossing the +650 hard stop. The uncommitted runtime draft was discarded and the tree +returned to the 484-line approved implementation checkpoint. Splitting typed +and database enforcement into separately mergeable PRs would leave an +incomplete security boundary, so the user approved a 1000-line hard stop +with the existing 500-line inspection, 120-character migration line gate, and +no SQL packing. Runtime work remains paused for exact-SHA amendment review. + ## 2026-07-14 - WS-AUTH-001-05A Repaired Contract Review Passed The exact contract at `7cc6058` passed senior engineering, architecture, product/ops, docs, reuse, QA, CI/test-delta, and security/privacy review. It locks closed envelope/reason/fact registries, cross-field project integrity, non-echoing Mapping/JSON admission, identical typed/direct-SQL behavior, -readable SQL, focused regressions, and the additions-plus-deletions 500/650 +readable SQL, focused regressions, and the then-approved 500/650 circuit breaker. Bounded runtime repair may proceed; AUTH-05B remains inactive. ## 2026-07-14 - WS-AUTH-001-05A Repaired Contract Review Failed diff --git a/.agent-loop/WORK_QUEUE.md b/.agent-loop/WORK_QUEUE.md index 2b74e70b..9a91d104 100644 --- a/.agent-loop/WORK_QUEUE.md +++ b/.agent-loop/WORK_QUEUE.md @@ -4,7 +4,7 @@ | Chunk | Title | Risk | Status | |---|---|---:|---| -| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | Repaired contract passed at `7cc6058`; bounded runtime repair and focused evidence | +| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | 1000-line circuit-breaker amendment review; runtime draft discarded at prior 650 stop | ## Planned Next diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md index 6c9dc4c4..4867c607 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md @@ -19,7 +19,7 @@ stopped. | `WS-AUTH-001-04A` | Request And Error Context | L1 | Merged through PR #111 as `90c9a28` | | `WS-AUTH-001-04B` | PostgreSQL Rate Controls | L1 | Merged through PR #113 as `05a63c8` | | `WS-AUTH-001-05` | Authority Evidence And Idempotency Foundation | L1 | Split before implementation into 05A and 05B | -| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | Repaired contract passed at `7cc6058`; runtime repair/evidence | +| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | 1000-line circuit-breaker amendment review | | `WS-AUTH-001-05B` | Authority Idempotency And Invalidation Foundation | L1 | Inactive pending 05A merge/memory and separate explicit start | | `WS-AUTH-001-06` | Canonical Actor Profile And Identity Link | L1 | Proposed | | `WS-AUTH-001-07` | Authorization Kernel And Permission Registry | L1 | Proposed | diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md index 23fa23e9..0e6b839b 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md @@ -48,8 +48,11 @@ explicitly started AUTH-05. Required L1 plan review rejected the combined audit/idempotency contract before runtime edits and required children 05A and 05B. The first repaired AUTH-05A contract passed at `7a9023b`, but exact-SHA implementation review demonstrated that closed privacy registries and readable -typed/SQL parity require a repaired 650-line contract. Contract re-review is -complete at `7cc6058`; bounded runtime repair and focused evidence are active. +typed/SQL parity require a larger readable contract. The repaired contract +passed at `7cc6058`, but its 650-line hard stop halted the first exact-registry +dry run at 689 lines before database parity was complete. A human-approved +1000-line ceiling amendment is under review; no runtime repair proceeds until +it passes. ## Active planning chunk diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md index b15b4c8c..d8f6482a 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md @@ -2,10 +2,10 @@ ## Status -Bounded runtime repair and focused evidence. Repaired contract review passed on -exact SHA `7cc6058` after the initial implementation review proved that safe -typed/SQL parity did not fit the original ceiling without compressing -security-sensitive SQL. +Circuit-breaker amendment review. Repaired contract review passed on exact SHA +`7cc6058`, but a readable exact-registry dry run crossed the then-current 650 +hard stop before PostgreSQL parity was complete. The draft was discarded; +runtime work is paused pending review of the human-approved 1000-line ceiling. ## Parent initiative @@ -21,7 +21,7 @@ event and existing task/checker behavior. - Risk: L1 / SLA P1. - Inspect scope at 500 changed non-comment production lines. -- Hard stop at 650 changed non-comment production lines, counting +- Hard stop at 1000 changed non-comment production lines, counting `backend/app/**` plus migration code; tests and evidence do not justify exceeding the production limit. - Security SQL must remain reviewable as named functions and multiline logical @@ -29,6 +29,9 @@ event and existing task/checker behavior. is prohibited. - This one-time ceiling repair is limited to the reviewer-required privacy and causation controls; it does not reactivate AUTH-05B scope. +- A readable typed-registry dry run reached 689 changed lines before the SQL + matrix was complete. The draft was discarded. Typed and database enforcement + are one atomic security boundary, so they are not split across mergeable PRs. ## Allowed files @@ -355,7 +358,7 @@ changed_production_lines=$(git diff --unified=0 origin/main...HEAD -- backend/ap awk '/^\+\+\+|^---/{next} /^[+-]/{line=substr($0,2); if (line !~ /^[[:space:]]*($|#)/) n++} END{print n+0}') printf 'AUTH-05A changed production lines: %s\n' "$changed_production_lines" if test "$changed_production_lines" -gt 500; then printf 'AUTH-05A inspection required\n'; fi -test "$changed_production_lines" -le 650 +test "$changed_production_lines" -le 1000 awk 'length($0) > 120 { print FNR ":" $0; bad=1 } END { exit bad }' \ backend/alembic/versions/0018_authority_audit_evidence.py if git diff --unified=0 origin/main...HEAD -- backend/tests | \ From 611abfcec7ded2ef49e899477cefae85432dc183 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 12:18:50 +0100 Subject: [PATCH 12/22] docs(auth): align AUTH-05A amendment state --- .../CHUNK_MAP.md | 5 +++-- .../WS-AUTH-001-workstream-authorization-service/STATUS.md | 6 +++--- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md index 4867c607..4a98a79d 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md @@ -99,5 +99,6 @@ approval passed. AUTH-04B post-merge memory then merged through PR #114 as rejected the combined contract before runtime edits and required 05A/05B. The first 05A implementation review proved the original 500-line ceiling incompatible with readable typed/database privacy parity. Repaired 05A contract -review passed at `7cc6058`; bounded runtime repair is active. 05B remains -inactive. Do not start AUTH-06 or POL-002-04. +review passed at `7cc6058`; the 1000-line circuit-breaker amendment is under +review and runtime is paused. 05B remains inactive. Do not start AUTH-06 or +POL-002-04. diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md index 0e6b839b..2c8ec7e4 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md @@ -79,7 +79,7 @@ None. | `WS-AUTH-001-04A` | Merged | `codex/ws-auth-001-04-request-api-controls` | #111 | Merged as `90c9a28`; production review `cdcaf77`; final branch head `36c4aa5`. | | `WS-AUTH-001-04B` | Merged | `codex/ws-auth-001-04b-postgres-rate-controls` | #113 | Merged as `05a63c8`; production review `67484b5`; final branch head `94fb2fe`. | | `WS-AUTH-001-05` | Split | `codex/ws-auth-001-05-authority-evidence` | - | Parent split before implementation into 05A and 05B. | -| `WS-AUTH-001-05A` | Runtime repair/evidence | `codex/ws-auth-001-05-authority-evidence` | - | Repaired contract passed at `7cc6058`; closed registries, non-echoing admission, typed/SQL parity, and readable SQL. | +| `WS-AUTH-001-05A` | Circuit-breaker amendment review | `codex/ws-auth-001-05-authority-evidence` | - | Runtime paused; human-approved 1000-line ceiling under exact-SHA review. | | `WS-AUTH-001-05B` | Inactive | - | - | Idempotency and invalidation; requires 05A merge/memory and separate explicit start. | | `WS-AUTH-001-06` | Proposed | - | - | Canonical actor profile and identity link. | | `WS-AUTH-001-07` | Proposed | - | - | Authorization kernel and permissions. | @@ -95,8 +95,8 @@ None. ## Blockers -AUTH-05A has no preimplementation blocker after repaired-contract review passed -at `7cc6058`. The combined AUTH-05 contract was +AUTH-05A runtime repair is paused until exact-SHA review passes the human-approved +1000-line circuit-breaker amendment. The combined AUTH-05 contract was rejected before code changes because migration `0017` is already owned by AUTH-04B and the shared-audit plus idempotency scope was not reviewable as one L1 change. Repaired AUTH-05A owns migration `0018`; AUTH-05B later owns From cd07e81989fe1fbb18815f37b8c8028d0d8c8f25 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 12:20:17 +0100 Subject: [PATCH 13/22] docs(auth): record AUTH-05A ceiling approval --- .agent-loop/LOOP_STATE.md | 4 ++-- .agent-loop/REVIEW_LOG.md | 9 +++++++++ .agent-loop/WORK_QUEUE.md | 2 +- .../CHUNK_MAP.md | 8 ++++---- .../STATUS.md | 10 +++++----- .../WS-AUTH-001-05A-shared-audit-authority-evidence.md | 7 +++---- 6 files changed, 24 insertions(+), 16 deletions(-) diff --git a/.agent-loop/LOOP_STATE.md b/.agent-loop/LOOP_STATE.md index 433aea5c..8824f0bf 100644 --- a/.agent-loop/LOOP_STATE.md +++ b/.agent-loop/LOOP_STATE.md @@ -16,8 +16,8 @@ - Prior `WS-AUTH-001-01` reviewed implementation SHA: `be0b836` - Prior `WS-AUTH-001-01` final merged branch head: `b5217e1` - Latest integrated `main` merge commit: `97cd0f5` -- Current gate: AUTH-05A 1000-line circuit-breaker amendment review after the - 650-line runtime dry run stopped without committing production edits. +- Current gate: AUTH-05A bounded runtime repair and focused evidence after the + 1000-line circuit-breaker amendment passed at `611abfc`. - Size checkpoint: AUTH-05A has a reviewer-required 500-line inspection and 1000-line production hard stop after the closed typed/SQL matrix dry run; security SQL may not be packed to satisfy it. diff --git a/.agent-loop/REVIEW_LOG.md b/.agent-loop/REVIEW_LOG.md index 8d797b4e..47b957d1 100644 --- a/.agent-loop/REVIEW_LOG.md +++ b/.agent-loop/REVIEW_LOG.md @@ -1,5 +1,14 @@ # Review Log +## 2026-07-14 - WS-AUTH-001-05A Circuit Breaker Amendment Passed + +The human-approved AUTH-05A-only 500-line inspection and 1000-line hard stop +passed all required exact-SHA reviewers at `611abfc`. Review confirmed the typed +and PostgreSQL privacy matrices must remain atomic, the incomplete 689-line +draft was discarded, SQL packing remains prohibited, focused behavior and 90 +percent subsystem coverage gates remain, and AUTH-05B stays inactive. Bounded +runtime repair may resume. + ## 2026-07-14 - WS-AUTH-001-05A Circuit Breaker Reopened The first post-approval exact-registry dry run reached 689 changed non-comment diff --git a/.agent-loop/WORK_QUEUE.md b/.agent-loop/WORK_QUEUE.md index 9a91d104..38d6552f 100644 --- a/.agent-loop/WORK_QUEUE.md +++ b/.agent-loop/WORK_QUEUE.md @@ -4,7 +4,7 @@ | Chunk | Title | Risk | Status | |---|---|---:|---| -| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | 1000-line circuit-breaker amendment review; runtime draft discarded at prior 650 stop | +| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | 1000-line amendment passed at `611abfc`; bounded runtime repair/evidence | ## Planned Next diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md index 4a98a79d..f7a2a0ad 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md @@ -19,7 +19,7 @@ stopped. | `WS-AUTH-001-04A` | Request And Error Context | L1 | Merged through PR #111 as `90c9a28` | | `WS-AUTH-001-04B` | PostgreSQL Rate Controls | L1 | Merged through PR #113 as `05a63c8` | | `WS-AUTH-001-05` | Authority Evidence And Idempotency Foundation | L1 | Split before implementation into 05A and 05B | -| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | 1000-line circuit-breaker amendment review | +| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | 1000-line amendment passed at `611abfc`; runtime repair/evidence | | `WS-AUTH-001-05B` | Authority Idempotency And Invalidation Foundation | L1 | Inactive pending 05A merge/memory and separate explicit start | | `WS-AUTH-001-06` | Canonical Actor Profile And Identity Link | L1 | Proposed | | `WS-AUTH-001-07` | Authorization Kernel And Permission Registry | L1 | Proposed | @@ -99,6 +99,6 @@ approval passed. AUTH-04B post-merge memory then merged through PR #114 as rejected the combined contract before runtime edits and required 05A/05B. The first 05A implementation review proved the original 500-line ceiling incompatible with readable typed/database privacy parity. Repaired 05A contract -review passed at `7cc6058`; the 1000-line circuit-breaker amendment is under -review and runtime is paused. 05B remains inactive. Do not start AUTH-06 or -POL-002-04. +review passed at `7cc6058`; the 1000-line circuit-breaker amendment passed at +`611abfc`, and bounded runtime repair is active. 05B remains inactive. Do not +start AUTH-06 or POL-002-04. diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md index 2c8ec7e4..fc3d57a9 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md @@ -51,8 +51,8 @@ implementation review demonstrated that closed privacy registries and readable typed/SQL parity require a larger readable contract. The repaired contract passed at `7cc6058`, but its 650-line hard stop halted the first exact-registry dry run at 689 lines before database parity was complete. A human-approved -1000-line ceiling amendment is under review; no runtime repair proceeds until -it passes. +1000-line ceiling amendment passed at `611abfc`; bounded runtime repair and +focused evidence are active. ## Active planning chunk @@ -79,7 +79,7 @@ None. | `WS-AUTH-001-04A` | Merged | `codex/ws-auth-001-04-request-api-controls` | #111 | Merged as `90c9a28`; production review `cdcaf77`; final branch head `36c4aa5`. | | `WS-AUTH-001-04B` | Merged | `codex/ws-auth-001-04b-postgres-rate-controls` | #113 | Merged as `05a63c8`; production review `67484b5`; final branch head `94fb2fe`. | | `WS-AUTH-001-05` | Split | `codex/ws-auth-001-05-authority-evidence` | - | Parent split before implementation into 05A and 05B. | -| `WS-AUTH-001-05A` | Circuit-breaker amendment review | `codex/ws-auth-001-05-authority-evidence` | - | Runtime paused; human-approved 1000-line ceiling under exact-SHA review. | +| `WS-AUTH-001-05A` | Runtime repair/evidence | `codex/ws-auth-001-05-authority-evidence` | - | Human-approved 1000-line ceiling passed at `611abfc`; bounded repair active. | | `WS-AUTH-001-05B` | Inactive | - | - | Idempotency and invalidation; requires 05A merge/memory and separate explicit start. | | `WS-AUTH-001-06` | Proposed | - | - | Canonical actor profile and identity link. | | `WS-AUTH-001-07` | Proposed | - | - | Authorization kernel and permissions. | @@ -95,8 +95,8 @@ None. ## Blockers -AUTH-05A runtime repair is paused until exact-SHA review passes the human-approved -1000-line circuit-breaker amendment. The combined AUTH-05 contract was +AUTH-05A has no preimplementation blocker after the human-approved 1000-line +circuit-breaker amendment passed at `611abfc`. The combined AUTH-05 contract was rejected before code changes because migration `0017` is already owned by AUTH-04B and the shared-audit plus idempotency scope was not reviewable as one L1 change. Repaired AUTH-05A owns migration `0018`; AUTH-05B later owns diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md index d8f6482a..fa67a12e 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md @@ -2,10 +2,9 @@ ## Status -Circuit-breaker amendment review. Repaired contract review passed on exact SHA -`7cc6058`, but a readable exact-registry dry run crossed the then-current 650 -hard stop before PostgreSQL parity was complete. The draft was discarded; -runtime work is paused pending review of the human-approved 1000-line ceiling. +Bounded runtime repair and focused evidence. Repaired contract review passed on +`7cc6058`; the readable exact-registry dry run stopped at the prior ceiling, and +the human-approved 1000-line amendment passed at `611abfc`. ## Parent initiative From a5559bdd229604c4353b78ebf515a50a6f9ac61b Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 12:48:05 +0100 Subject: [PATCH 14/22] fix(auth): enforce closed authority evidence registries --- .../versions/0018_authority_audit_evidence.py | 477 ++++++++++++++++-- backend/app/modules/audit/schemas.py | 396 ++++++++++----- backend/tests/test_alembic.py | 15 +- backend/tests/test_audit.py | 279 +++++++--- 4 files changed, 916 insertions(+), 251 deletions(-) diff --git a/backend/alembic/versions/0018_authority_audit_evidence.py b/backend/alembic/versions/0018_authority_audit_evidence.py index 10db9ed7..aa208506 100644 --- a/backend/alembic/versions/0018_authority_audit_evidence.py +++ b/backend/alembic/versions/0018_authority_audit_evidence.py @@ -10,8 +10,7 @@ revision = "0018_authority_audit_evidence" down_revision = "0017_api_controls" -branch_labels = None -depends_on = None +branch_labels = depends_on = None AUTHORITY_EVENT_TYPES = ( "ActorProfileProvisioned", @@ -35,6 +34,51 @@ "SensitiveAuthorizationDenied", "AuthorityInvalidationRequested", ) +PERMISSIONS = """actor.profile.read_self actor.profile.update_self actor.profile.read_any +actor.profile.suspend actor.profile.reactivate actor.profile.deactivate actor.identity_link.read +actor.identity_link.revoke actor.identity_link.reactivate actor.service.provision admin_role.read +admin_role.grant admin_role.revoke project.create project.read project.update project.archive +project.guide.manage project.effective_policy.manage project.task.manage project.review_policy.manage +project.role_grant.read project.role_grant.manage task.queue.read task.claim submission.create +submission.read_own submission.read_for_review review.queue.read review.queue.inspect review.claim +review.release review.decline_preference review.decision review.lease.force_release review.chain.read +contribution.read_self contribution.read_project compensation.policy.manage +compensation.adapter_binding.manage compensation.award.read compensation.delivery.reconcile +operations.status.read operations.timer.run operations.reconcile.run operations.outbox.retry +operations.projection.rebuild audit.read audit.export""".split() +DENIAL_CODES = """required_scope_missing unsupported_subject_kind service_actor_not_provisioned +identity_link_revoked actor_suspended actor_deactivated permission_not_granted scope_not_authorized +self_grant_forbidden self_role_revoke_forbidden resource_guard_denied actor_not_found grant_not_found +resource_not_found actor_already_suspended actor_not_suspended actor_deactivated_terminal +last_access_administrator admin_role_grant_exists project_role_grant_exists identity_link_conflict +resource_project_mismatch idempotency_mismatch invalid_role_scope invalid_project_role +qualification_snapshot_invalid""".split() +REASONS = { + "ActorProfileProvisioned": ("automatic_first_access",), + "ServiceActorProvisioned": ("manual_service_provisioning",), + "ActorIdentityLinked": ("identity_lifecycle_change",), + "ActorIdentityLinkRevoked": ("identity_lifecycle_change",), + "ActorIdentityLinkReactivated": ("identity_lifecycle_change",), + "ActorProfileSuspended": ("security_response", "administrative_correction"), + "ActorProfileReactivated": ("administrative_correction",), + "ActorProfileDeactivated": ("security_response", "administrative_correction"), + "InitialAccessAdministratorBootstrapped": ("initial_access_bootstrap",), + "AdminRoleGrantIssued": ("authority_assignment",), + "AdminRoleGrantRevoked": ("authority_revocation",), + "AdminRoleGrantIssueDenied": ("authorization_policy_denial",), + "LastAccessAdministratorOperationDenied": ("authorization_policy_denial",), + "ProjectRoleQualificationSnapshotCaptured": ("qualification_evidence_captured",), + "ProjectRoleGrantIssued": ("authority_assignment",), + "ProjectRoleGrantReplaced": ("authority_replacement",), + "ProjectRoleGrantRevoked": ("authority_revocation",), + "SensitiveAuthorizationAllowed": ("authorization_evaluation",), + "SensitiveAuthorizationDenied": ("authorization_evaluation",), + "AuthorityInvalidationRequested": ("authority_state_changed",), +} + + +def _sql_tokens(values) -> str: + return ", ".join(f"'{value}'" for value in values) def upgrade() -> None: @@ -72,27 +116,234 @@ def upgrade() -> None: op.add_column("audit_events", column) op.alter_column("audit_events", "external_subject", existing_type=sa.String(200), nullable=True) op.alter_column("audit_events", "external_issuer", existing_type=sa.String(200), nullable=True) - op.execute("""create function authority_facts_are_safe(facts json) returns boolean language sql immutable strict as $$ - select json_typeof(facts) = 'object' and not exists (select 1 from json_each(facts) item where - item.key not in ('status','subject_kind','provisioning_method','link_status','role_id','scope_type','scope_id','effective','qualification_status','decision_code','target_status') - or json_typeof(item.value) not in ('string','boolean','null') - or (item.key = 'effective' and json_typeof(item.value) not in ('boolean','null')) - or (item.key <> 'effective' and json_typeof(item.value) not in ('string','null')) - or (json_typeof(item.value) = 'string' and ((item.value #>> '{}') ~ '^eyJ' or (item.key in ('role_id','scope_id') and (item.value #>> '{}') !~ '^[A-Za-z0-9][A-Za-z0-9_.:-]{0,119}$') or (item.key not in ('role_id','scope_id') and (item.value #>> '{}') !~ '^[a-z][a-z0-9_]{0,79}$')))) $$""") + op.execute( + """ + create function authority_facts_are_safe(facts json) + returns boolean language sql immutable strict as $$ + select json_typeof(facts) = 'object' + and (select count(*) from json_each(facts)) <= 8 + and not exists ( + select 1 from json_each(facts) item + where item.key not in ( + 'status', 'subject_kind', 'provisioning_method', 'role', + 'scope_type', 'scope_id', 'effective', 'allowed' + ) + or case item.key + when 'status' then item.value #>> '{}' not in ( + 'active', 'suspended', 'deactivated', 'revoked', 'captured' + ) + when 'subject_kind' then item.value #>> '{}' not in ('human', 'service') + when 'provisioning_method' then item.value #>> '{}' not in ( + 'automatic_first_access', 'manual_service_provisioning' + ) + when 'role' then item.value #>> '{}' not in ( + 'access_administrator', 'operator', 'project_manager', + 'finance_authority', 'audit_authority', 'submitter', 'reviewer', 'both' + ) + when 'scope_type' then item.value #>> '{}' not in ('system', 'project') + when 'scope_id' then (item.value #>> '{}') !~ + '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$' + when 'effective' then json_typeof(item.value) <> 'boolean' + when 'allowed' then json_typeof(item.value) <> 'boolean' + else true + end + ) + $$ + """ + ) + op.execute( + """ + create function authority_grant_facts_are_safe( + facts json, roles text[], expected_status text, + expected_effective boolean, envelope_project_id text + ) returns boolean language sql immutable strict as $$ + select authority_facts_are_safe(facts) + and facts->>'role' = any(roles) + and facts->>'status' = expected_status + and (facts->>'effective')::boolean = expected_effective + and ( + ( + facts->>'scope_type' = 'system' + and envelope_project_id is null + and not facts::jsonb ? 'scope_id' + and facts->>'role' not in ('submitter', 'reviewer', 'both') + and (select count(*) from json_each(facts)) = 4 + ) or ( + facts->>'scope_type' = 'project' + and envelope_project_id is not null + and facts->>'scope_id' = envelope_project_id + and facts->>'role' not in ('access_administrator', 'operator') + and (select count(*) from json_each(facts)) = 5 + ) + ) + $$ + """ + ) + op.execute( + """ + create function authority_event_facts_are_safe( + event_name text, before_state json, after_state json, envelope_project_id text + ) returns boolean language plpgsql immutable as $$ + begin + if (before_state is not null and not authority_facts_are_safe(before_state)) + or (after_state is not null and not authority_facts_are_safe(after_state)) then + return false; + end if; + case event_name + when 'ActorProfileProvisioned' then + return before_state is null and after_state::jsonb = + '{"status":"active","subject_kind":"human",' + '"provisioning_method":"automatic_first_access"}'::jsonb; + when 'ServiceActorProvisioned' then + return before_state is null and after_state::jsonb = + '{"status":"active","subject_kind":"service",' + '"provisioning_method":"manual_service_provisioning"}'::jsonb; + when 'ActorIdentityLinked' then + return before_state is null and after_state::jsonb in ( + '{"status":"active","subject_kind":"human"}'::jsonb, + '{"status":"active","subject_kind":"service"}'::jsonb + ); + when 'ActorIdentityLinkRevoked' then + return before_state::jsonb = '{"status":"active"}'::jsonb + and after_state::jsonb = '{"status":"revoked"}'::jsonb; + when 'ActorIdentityLinkReactivated' then + return before_state::jsonb = '{"status":"revoked"}'::jsonb + and after_state::jsonb = '{"status":"active"}'::jsonb; + when 'ActorProfileSuspended' then + return before_state::jsonb = '{"status":"active"}'::jsonb + and after_state::jsonb = '{"status":"suspended"}'::jsonb; + when 'ActorProfileReactivated' then + return before_state::jsonb = '{"status":"suspended"}'::jsonb + and after_state::jsonb = '{"status":"active"}'::jsonb; + when 'ActorProfileDeactivated' then + return before_state::jsonb in ( + '{"status":"active"}'::jsonb, '{"status":"suspended"}'::jsonb + ) and after_state::jsonb = '{"status":"deactivated"}'::jsonb; + when 'InitialAccessAdministratorBootstrapped' then + return before_state is null and authority_grant_facts_are_safe( + after_state, array['access_administrator'], 'active', true, null + ); + when 'AdminRoleGrantIssued' then + return before_state is null and authority_grant_facts_are_safe( + after_state, + array[ + 'access_administrator', 'operator', 'project_manager', + 'finance_authority', 'audit_authority' + ], 'active', true, envelope_project_id + ); + when 'ProjectRoleGrantIssued' then + return before_state is null and authority_grant_facts_are_safe( + after_state, array['submitter', 'reviewer', 'both'], + 'active', true, envelope_project_id + ); + when 'AdminRoleGrantRevoked', 'ProjectRoleGrantRevoked' then + return authority_grant_facts_are_safe( + before_state, + case when event_name = 'AdminRoleGrantRevoked' then + array[ + 'access_administrator', 'operator', 'project_manager', + 'finance_authority', 'audit_authority' + ] + else array['submitter', 'reviewer', 'both'] end, + 'active', true, envelope_project_id + ) and authority_grant_facts_are_safe( + after_state, + case when event_name = 'AdminRoleGrantRevoked' then + array[ + 'access_administrator', 'operator', 'project_manager', + 'finance_authority', 'audit_authority' + ] + else array['submitter', 'reviewer', 'both'] end, + 'revoked', false, envelope_project_id + ) and before_state->>'role' = after_state->>'role' + and before_state->>'scope_type' = after_state->>'scope_type' + and coalesce(before_state->>'scope_id', '') = + coalesce(after_state->>'scope_id', ''); + when 'ProjectRoleGrantReplaced' then + return authority_grant_facts_are_safe( + before_state, array['submitter', 'reviewer', 'both'], + 'active', true, envelope_project_id + ) and authority_grant_facts_are_safe( + after_state, array['submitter', 'reviewer', 'both'], + 'active', true, envelope_project_id + ) and before_state->>'scope_id' = after_state->>'scope_id'; + when 'ProjectRoleQualificationSnapshotCaptured' then + return before_state is null + and after_state::jsonb = '{"status":"captured"}'::jsonb; + when 'AdminRoleGrantIssueDenied', 'LastAccessAdministratorOperationDenied' then + return before_state is null and after_state is null; + when 'SensitiveAuthorizationAllowed' then + return before_state is null and after_state::jsonb = '{"allowed": true}'::jsonb; + when 'SensitiveAuthorizationDenied' then + return before_state is null and after_state::jsonb = '{"allowed": false}'::jsonb; + when 'AuthorityInvalidationRequested' then + return before_state::jsonb = '{"effective": true}'::jsonb + and after_state::jsonb = '{"effective": false}'::jsonb; + else return false; + end case; + end + $$ + """ + ) - event_tokens = ", ".join(f"'{value}'" for value in AUTHORITY_EVENT_TYPES) + event_tokens = _sql_tokens(AUTHORITY_EVENT_TYPES) + permission_tokens = _sql_tokens(PERMISSIONS) + denial_tokens = _sql_tokens(DENIAL_CODES) + reason_rules = " or ".join( + f"(event_type = '{event}' and reason in ({_sql_tokens(reasons)}))" + for event, reasons in REASONS.items() + ) + entity_tokens = _sql_tokens( + ( + "actor_profile", + "actor_identity_link", + "admin_role_grant", + "qualification_snapshot", + "project_role_grant", + "authorization_decision", + "authority_invalidation", + ) + ) + resource_tokens = _sql_tokens( + """actor_profile actor_identity_link admin_role_grant project project_role_grant task + submission review contribution compensation_award compensation_delivery operations + audit_event""".split() + ) + uuid_target_tokens = _sql_tokens( + ( + "actor_profile", + "actor_identity_link", + "admin_role_grant", + "qualification_snapshot", + "project_role_grant", + ) + ) + uuid_pattern = r"^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$" op.create_check_constraint( "domain_shape", "audit_events", """ - (event_domain = 'legacy_lifecycle' and event_version is null and occurred_at is null and actor_ref_kind is null and request_id is null and correlation_id is null - and target_actor_ref_kind is null and target_actor_ref is null and matched_grant_id is null and permission_id is null and project_id is null - and resource_type is null and resource_id is null and target_ref_kind is null and target_ref_id is null and denial_code is null - and idempotency_reference is null and invalidation_cause_event_id is null and invalidation_target_kind is null and invalidation_target_ref is null - and before_facts is null and after_facts is null and external_subject is not null and external_issuer is not null) - or (event_domain = 'authority' and event_version = 1 and occurred_at is not null and actor_ref_kind in ('legacy_actor','actor_profile','system_principal') - and request_id is not null and correlation_id is not null and from_status is null and to_status is null and (reason is null or length(reason) <= 1000) - and external_subject is null and external_issuer is null and actor_roles::jsonb = '[]'::jsonb and claim_snapshot::jsonb = '{}'::jsonb and auth_source = 'local_authority' and is_dev_auth = false and event_payload::jsonb = '{}'::jsonb) + ( + event_domain = 'legacy_lifecycle' + and event_version is null and occurred_at is null and actor_ref_kind is null + and request_id is null and correlation_id is null and target_actor_ref_kind is null + and target_actor_ref is null and matched_grant_id is null and permission_id is null + and project_id is null and resource_type is null and resource_id is null + and target_ref_kind is null and target_ref_id is null and denial_code is null + and idempotency_reference is null and invalidation_cause_event_id is null + and invalidation_target_kind is null and invalidation_target_ref is null + and before_facts is null and after_facts is null + and external_subject is not null and external_issuer is not null + ) or ( + event_domain = 'authority' and event_version = 1 and occurred_at is not null + and actor_ref_kind in ('legacy_actor', 'actor_profile', 'system_principal') + and request_id is not null and correlation_id is not null + and from_status is null and to_status is null and reason is not null + and external_subject is null and external_issuer is null + and actor_roles::jsonb = '[]'::jsonb and claim_snapshot::jsonb = '{}'::jsonb + and auth_source = 'local_authority' and is_dev_auth = false + and event_payload::jsonb = '{}'::jsonb + ) """, ) op.create_check_constraint( @@ -100,31 +351,86 @@ def upgrade() -> None: "audit_events", f"event_domain <> 'authority' or event_type in ({event_tokens})", ) + op.create_check_constraint( + "authority_registries", + "audit_events", + f""" + event_domain <> 'authority' or ( + reason is not null and ({reason_rules}) + and (permission_id is null or permission_id in ({permission_tokens})) + and (denial_code is null or denial_code in ({denial_tokens})) + ) + """, + ) op.create_check_constraint( "reference_pairs", "audit_events", """ - (target_actor_ref_kind is null) = (target_actor_ref is null) and (resource_type is null) = (resource_id is null) - and (target_ref_kind is null) = (target_ref_id is null) and (invalidation_target_kind is null) = (invalidation_target_ref is null) + (target_actor_ref_kind is null) = (target_actor_ref is null) + and (resource_type is not null or resource_id is null) + and (target_ref_kind is null) = (target_ref_id is null) + and (invalidation_target_kind is null) = (invalidation_target_ref is null) and (invalidation_cause_event_id is null or invalidation_cause_event_id <> id) """, ) op.create_check_constraint( "authority_privacy_bounds", "audit_events", - """ - event_domain <> 'authority' or ((((actor_ref_kind in ('legacy_actor','actor_profile') and actor_id ~ '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$') - or (actor_ref_kind = 'system_principal' and actor_id ~ '^[a-z][a-z0-9_]*(:[a-z][a-z0-9_-]*)+$')) - and (target_actor_ref is null or ((target_actor_ref_kind in ('legacy_actor','actor_profile') and target_actor_ref ~ '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$') - or (target_actor_ref_kind = 'system_principal' and target_actor_ref ~ '^[a-z][a-z0-9_]*(:[a-z][a-z0-9_-]*)+$'))) - and entity_id ~ '^[A-Za-z0-9][A-Za-z0-9_.:-]{0,35}$' and concat_ws('',matched_grant_id,permission_id,project_id,resource_type,resource_id,target_ref_kind,target_ref_id,denial_code,invalidation_target_kind,invalidation_target_ref) !~ '[/@[:space:]]' - and (reason is null or reason ~ '^[A-Za-z0-9][A-Za-z0-9 _.,:;()!?-]*$'))) + f""" + event_domain <> 'authority' or ( + entity_type in ({entity_tokens}) and entity_id ~ '{uuid_pattern}' + and ( + (actor_ref_kind in ('legacy_actor', 'actor_profile') and actor_id ~ '{uuid_pattern}') + or (actor_ref_kind = 'system_principal' and actor_id = 'workstream:system:bootstrap') + ) + and ( + target_actor_ref is null + or (target_actor_ref_kind = 'actor_profile' and target_actor_ref ~ '{uuid_pattern}') + ) + and (matched_grant_id is null or matched_grant_id ~ '{uuid_pattern}') + and (project_id is null or project_id ~ '{uuid_pattern}') + and (resource_type is null or resource_type in ({resource_tokens})) + and (resource_id is null or resource_id ~ '{uuid_pattern}') + and ( + target_ref_kind is null + or (target_ref_kind in ({uuid_target_tokens}) and target_ref_id ~ '{uuid_pattern}') + or (target_ref_kind = 'permission_registry' and target_ref_id in ({permission_tokens})) + ) + and ( + invalidation_target_kind is null + or ( + invalidation_target_kind in ({uuid_target_tokens}) + and invalidation_target_ref ~ '{uuid_pattern}' + ) + or ( + invalidation_target_kind = 'permission_registry' + and invalidation_target_ref in ({permission_tokens}) + ) + ) + and ( + entity_type not in ('authorization_decision', 'authority_invalidation') + or entity_id = id + ) + and ( + resource_type <> 'project' or resource_id is null + or (project_id is not null and resource_id = project_id) + ) + ) """, ) op.create_check_constraint( "fact_bounds", "audit_events", - "(before_facts is null or (authority_facts_are_safe(before_facts) and octet_length(before_facts::text) <= 4096)) and (after_facts is null or (authority_facts_are_safe(after_facts) and octet_length(after_facts::text) <= 4096))", + """ + event_domain <> 'authority' or ( + (before_facts is null or octet_length(before_facts::text) <= 4096) + and (after_facts is null or octet_length(after_facts::text) <= 4096) + and coalesce( + authority_event_facts_are_safe(event_type, before_facts, after_facts, project_id), + false + ) + ) + """, ) op.create_foreign_key( "fk_audit_events_invalidation_cause", @@ -137,10 +443,26 @@ def upgrade() -> None: "foundation_shapes", "audit_events", """ - event_domain <> 'authority' or event_type not in ('SensitiveAuthorizationAllowed','SensitiveAuthorizationDenied','AuthorityInvalidationRequested') - or (event_type = 'SensitiveAuthorizationAllowed' and permission_id is not null and denial_code is null and invalidation_cause_event_id is null and invalidation_target_kind is null) - or (event_type = 'SensitiveAuthorizationDenied' and permission_id is not null and denial_code is not null and invalidation_cause_event_id is null and invalidation_target_kind is null and idempotency_reference is null) - or (event_type = 'AuthorityInvalidationRequested' and invalidation_cause_event_id is not null and invalidation_target_kind is not null and denial_code is null) + event_domain <> 'authority' + or event_type not in ( + 'SensitiveAuthorizationAllowed', 'SensitiveAuthorizationDenied', + 'AuthorityInvalidationRequested' + ) + or ( + event_type = 'SensitiveAuthorizationAllowed' and permission_id is not null + and denial_code is null and invalidation_cause_event_id is null + and invalidation_target_kind is null + ) + or ( + event_type = 'SensitiveAuthorizationDenied' and permission_id is not null + and denial_code is not null and invalidation_cause_event_id is null + and invalidation_target_kind is null and idempotency_reference is null + ) + or ( + event_type = 'AuthorityInvalidationRequested' + and invalidation_cause_event_id is not null + and invalidation_target_kind is not null and denial_code is null + ) """, ) for name, fields in ( @@ -152,16 +474,58 @@ def upgrade() -> None: ): op.create_index(name, "audit_events", fields) - op.execute("""create function set_authority_audit_database_time() returns trigger language plpgsql as $$ begin - if new.event_domain = 'authority' then - if new.invalidation_cause_event_id is not null and not exists (select 1 from audit_events where id = new.invalidation_cause_event_id and event_domain = 'authority') then - raise exception 'invalid authority invalidation cause' using errcode = '23503'; end if; - new.occurred_at = statement_timestamp(); - else new.occurred_at = null; end if; return new; end $$""") - op.execute("create trigger audit_events_set_authority_time before insert on audit_events for each row execute function set_authority_audit_database_time()") - op.execute("""create function reject_audit_event_mutation() returns trigger language plpgsql as $$ begin raise exception 'audit events are append-only' using errcode = '55000'; end $$""") - op.execute("create trigger audit_events_reject_update_delete before update or delete on audit_events for each row execute function reject_audit_event_mutation()") - op.execute("create trigger audit_events_reject_truncate before truncate on audit_events for each statement execute function reject_audit_event_mutation()") + op.execute( + """ + create function set_authority_audit_database_time() + returns trigger language plpgsql as $$ + begin + if new.event_domain = 'authority' then + if new.invalidation_cause_event_id is not null and not exists ( + select 1 from audit_events + where id = new.invalidation_cause_event_id and event_domain = 'authority' + ) then + raise exception 'invalid authority invalidation cause' using errcode = '23503'; + end if; + new.occurred_at = statement_timestamp(); + else + new.occurred_at = null; + end if; + return new; + end + $$ + """ + ) + op.execute( + """ + create trigger audit_events_set_authority_time + before insert on audit_events for each row + execute function set_authority_audit_database_time() + """ + ) + op.execute( + """ + create function reject_audit_event_mutation() + returns trigger language plpgsql as $$ + begin + raise exception 'audit events are append-only' using errcode = '55000'; + end + $$ + """ + ) + op.execute( + """ + create trigger audit_events_reject_update_delete + before update or delete on audit_events for each row + execute function reject_audit_event_mutation() + """ + ) + op.execute( + """ + create trigger audit_events_reject_truncate + before truncate on audit_events for each statement + execute function reject_audit_event_mutation() + """ + ) def downgrade() -> None: @@ -196,18 +560,37 @@ def downgrade() -> None: "fact_bounds", "authority_privacy_bounds", "reference_pairs", + "authority_registries", "authority_tokens", "domain_shape", ): op.drop_constraint(name, "audit_events", type_="check") + op.execute("drop function authority_event_facts_are_safe(text, json, json, text)") + op.execute("drop function authority_grant_facts_are_safe(json, text[], text, boolean, text)") op.execute("drop function authority_facts_are_safe(json)") for column in reversed( ( - "event_version", "occurred_at", "actor_ref_kind", "request_id", "correlation_id", - "target_actor_ref_kind", "target_actor_ref", "matched_grant_id", "permission_id", - "project_id", "resource_type", "resource_id", "target_ref_kind", "target_ref_id", - "denial_code", "idempotency_reference", "invalidation_cause_event_id", - "invalidation_target_kind", "invalidation_target_ref", "before_facts", "after_facts", + "event_version", + "occurred_at", + "actor_ref_kind", + "request_id", + "correlation_id", + "target_actor_ref_kind", + "target_actor_ref", + "matched_grant_id", + "permission_id", + "project_id", + "resource_type", + "resource_id", + "target_ref_kind", + "target_ref_id", + "denial_code", + "idempotency_reference", + "invalidation_cause_event_id", + "invalidation_target_kind", + "invalidation_target_ref", + "before_facts", + "after_facts", "event_domain", ) ): diff --git a/backend/app/modules/audit/schemas.py b/backend/app/modules/audit/schemas.py index 2220e69b..142fc4af 100644 --- a/backend/app/modules/audit/schemas.py +++ b/backend/app/modules/audit/schemas.py @@ -2,52 +2,68 @@ from __future__ import annotations +from collections.abc import Mapping from enum import StrEnum import json -import re -from typing import Annotated, Self -from uuid import UUID, uuid4 - -from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator - -_FACT_KEY = re.compile(r"^[a-z][a-z0-9_]{0,63}$") -_SAFE_REF = re.compile(r"^[A-Za-z0-9][A-Za-z0-9_.:-]{0,119}$") -_SYSTEM_REF = re.compile(r"^[a-z][a-z0-9_]*(?::[a-z][a-z0-9_-]*)+$") -_SAFE_REASON = re.compile(r"^[A-Za-z0-9][A-Za-z0-9 _.,:;()!?-]{0,999}$") -_FACT_BOOL_KEYS = frozenset({"effective"}) -_FACT_ID_KEYS = frozenset({"role_id", "scope_id"}) -_FACT_TOKEN = re.compile(r"^[a-z][a-z0-9_]{0,79}$") -SafeRef100 = Annotated[str, Field(max_length=100, pattern=_SAFE_REF.pattern)] -SafeRef36 = Annotated[str, Field(max_length=36, pattern=_SAFE_REF.pattern)] -SafeToken = Annotated[str, Field(max_length=80, pattern=r"^[a-z][a-z0-9_.:-]*$")] -SafeToken32 = Annotated[str, Field(max_length=32, pattern=r"^[a-z][a-z0-9_.:-]*$")] - - -def _is_uuid(value: str) -> bool: - """Return whether a value is a canonical local UUID.""" - try: - return str(UUID(value)) == value - except ValueError: - return False +from typing import Annotated, Any, Self +from uuid import UUID +from pydantic import BaseModel, ConfigDict, Field, model_validator -def _safe_fact_value(key: str, value: object) -> bool: - """Accept only typed booleans or bounded local tokens in state facts.""" - if value is None: - return True - if key in _FACT_BOOL_KEYS: - return type(value) is bool - pattern = _SAFE_REF if key in _FACT_ID_KEYS else _FACT_TOKEN - return ( - isinstance(value, str) - and not value.startswith("eyJ") - and pattern.fullmatch(value) is not None - ) - - -def _safe_actor_ref(kind: ActorReferenceKind, value: str) -> bool: - """Validate one local or registered system actor reference.""" - return bool(_SYSTEM_REF.fullmatch(value)) if kind == ActorReferenceKind.SYSTEM_PRINCIPAL else _is_uuid(value) +_ENTITY_TYPES = frozenset( + { + "actor_profile", + "actor_identity_link", + "admin_role_grant", + "qualification_snapshot", + "project_role_grant", + "authorization_decision", + "authority_invalidation", + } +) +_RESOURCE_TYPES = frozenset( + """actor_profile actor_identity_link admin_role_grant project project_role_grant task + submission review contribution compensation_award compensation_delivery operations + audit_event""".split() +) +_UUID_TARGET_KINDS = frozenset( + {"actor_profile", "actor_identity_link", "admin_role_grant", "qualification_snapshot", "project_role_grant"} +) +_PERMISSIONS = frozenset( + """actor.profile.read_self actor.profile.update_self actor.profile.read_any + actor.profile.suspend actor.profile.reactivate actor.profile.deactivate + actor.identity_link.read actor.identity_link.revoke actor.identity_link.reactivate + actor.service.provision admin_role.read admin_role.grant admin_role.revoke project.create + project.read project.update project.archive project.guide.manage project.effective_policy.manage + project.task.manage project.review_policy.manage project.role_grant.read + project.role_grant.manage task.queue.read task.claim submission.create submission.read_own + submission.read_for_review review.queue.read review.queue.inspect review.claim review.release + review.decline_preference review.decision review.lease.force_release review.chain.read + contribution.read_self contribution.read_project compensation.policy.manage + compensation.adapter_binding.manage compensation.award.read compensation.delivery.reconcile + operations.status.read operations.timer.run operations.reconcile.run operations.outbox.retry + operations.projection.rebuild audit.read audit.export""".split() +) +_DENIAL_CODES = frozenset( + """required_scope_missing unsupported_subject_kind service_actor_not_provisioned + identity_link_revoked actor_suspended actor_deactivated permission_not_granted + scope_not_authorized self_grant_forbidden self_role_revoke_forbidden resource_guard_denied + actor_not_found grant_not_found resource_not_found actor_already_suspended actor_not_suspended + actor_deactivated_terminal last_access_administrator admin_role_grant_exists + project_role_grant_exists identity_link_conflict resource_project_mismatch idempotency_mismatch + invalid_role_scope invalid_project_role qualification_snapshot_invalid""".split() +) +_ADMIN_ROLES = frozenset( + {"access_administrator", "operator", "project_manager", "finance_authority", "audit_authority"} +) +_PROJECT_ROLES = frozenset({"submitter", "reviewer", "both"}) +_FACT_VALUES: dict[str, frozenset[str]] = { + "status": frozenset({"active", "suspended", "deactivated", "revoked", "captured"}), + "subject_kind": frozenset({"human", "service"}), + "provisioning_method": frozenset({"automatic_first_access", "manual_service_provisioning"}), + "role": _ADMIN_ROLES | _PROJECT_ROLES, + "scope_type": frozenset({"system", "project"}), +} class AuthorityEventType(StrEnum): @@ -75,25 +91,6 @@ class AuthorityEventType(StrEnum): AUTHORITY_INVALIDATION_REQUESTED = "AuthorityInvalidationRequested" -# Specification categories are contiguous in the closed enum above. -_EVENT_TYPES = tuple(AuthorityEventType) -_EVENT_FACT_KEYS = { - **dict.fromkeys( - _EVENT_TYPES[:8], - frozenset({"status", "subject_kind", "provisioning_method", "link_status"}), - ), - **dict.fromkeys( - _EVENT_TYPES[8:17], - frozenset( - {"status", "role_id", "scope_type", "scope_id", "effective", "qualification_status"} - ), - ), - **dict.fromkeys(_EVENT_TYPES[17:19], frozenset({"status", "decision_code", "effective"})), - _EVENT_TYPES[19]: frozenset({"status", "target_status", "effective"}), -} -_ALL_FACT_KEYS = frozenset().union(*_EVENT_FACT_KEYS.values()) - - class ActorReferenceKind(StrEnum): """Stable namespaces usable before and after canonical actor migration.""" @@ -102,99 +99,242 @@ class ActorReferenceKind(StrEnum): SYSTEM_PRINCIPAL = "system_principal" +_REASONS = { + AuthorityEventType.ACTOR_PROFILE_PROVISIONED: {"automatic_first_access"}, + AuthorityEventType.SERVICE_ACTOR_PROVISIONED: {"manual_service_provisioning"}, + **dict.fromkeys(tuple(AuthorityEventType)[2:5], {"identity_lifecycle_change"}), + AuthorityEventType.ACTOR_PROFILE_SUSPENDED: {"security_response", "administrative_correction"}, + AuthorityEventType.ACTOR_PROFILE_REACTIVATED: {"administrative_correction"}, + AuthorityEventType.ACTOR_PROFILE_DEACTIVATED: {"security_response", "administrative_correction"}, + AuthorityEventType.INITIAL_ACCESS_ADMIN_BOOTSTRAPPED: {"initial_access_bootstrap"}, + AuthorityEventType.ADMIN_ROLE_GRANT_ISSUED: {"authority_assignment"}, + AuthorityEventType.ADMIN_ROLE_GRANT_REVOKED: {"authority_revocation"}, + AuthorityEventType.ADMIN_ROLE_GRANT_ISSUE_DENIED: {"authorization_policy_denial"}, + AuthorityEventType.LAST_ACCESS_ADMIN_OPERATION_DENIED: {"authorization_policy_denial"}, + AuthorityEventType.PROJECT_ROLE_QUALIFICATION_CAPTURED: {"qualification_evidence_captured"}, + AuthorityEventType.PROJECT_ROLE_GRANT_ISSUED: {"authority_assignment"}, + AuthorityEventType.PROJECT_ROLE_GRANT_REPLACED: {"authority_replacement"}, + AuthorityEventType.PROJECT_ROLE_GRANT_REVOKED: {"authority_revocation"}, + AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED: {"authorization_evaluation"}, + AuthorityEventType.SENSITIVE_AUTHORIZATION_DENIED: {"authorization_evaluation"}, + AuthorityEventType.AUTHORITY_INVALIDATION_REQUESTED: {"authority_state_changed"}, +} + + +def _enum_value(value: object, enum: type[StrEnum]) -> str | None: + raw = value.value if isinstance(value, enum) else value + return raw if isinstance(raw, str) and raw in {item.value for item in enum} else None + + +def _uuid(value: object) -> str | None: + raw = str(value) if isinstance(value, UUID) else value + try: + return raw if isinstance(raw, str) and str(UUID(raw)) == raw else None + except ValueError: + return None + + +def _registered(value: object, values: frozenset[str] | set[str]) -> bool: + return isinstance(value, str) and value in values + + +def _facts(value: object) -> dict[str, object] | None: + if value is None: + return None + if not isinstance(value, Mapping) or len(value) > 8: + return None + if not set(value).issubset(frozenset(_FACT_VALUES) | {"effective", "allowed", "scope_id"}): + return None + data = dict(value) + for key, item in data.items(): + if key in {"effective", "allowed"} and type(item) is not bool: + return None + if key == "scope_id" and _uuid(item) is None: + return None + if key in _FACT_VALUES and (not isinstance(item, str) or item not in _FACT_VALUES[key]): + return None + if len(json.dumps(data, separators=(",", ":"), ensure_ascii=False).encode()) > 4096: + return None + return data + + +def _grant_facts(facts: dict[str, object] | None, roles: frozenset[str], status: str, effective: bool) -> bool: + if facts is None or facts.get("role") not in roles: + return False + scope = facts.get("scope_type") + expected = {"status", "role", "scope_type", "effective"} | ({"scope_id"} if scope == "project" else set()) + return ( + set(facts) == expected + and facts["status"] == status + and facts["effective"] is effective + and scope in {"system", "project"} + and not (facts["role"] in {"access_administrator", "operator"} and scope != "system") + and not (facts["role"] in _PROJECT_ROLES and scope != "project") + ) + + +def _event_facts_valid(event: AuthorityEventType, before: dict[str, object] | None, after: dict[str, object] | None) -> bool: + exact = { + AuthorityEventType.ACTOR_PROFILE_PROVISIONED: (None, {"status": "active", "subject_kind": "human", "provisioning_method": "automatic_first_access"}), + AuthorityEventType.SERVICE_ACTOR_PROVISIONED: (None, {"status": "active", "subject_kind": "service", "provisioning_method": "manual_service_provisioning"}), + AuthorityEventType.ACTOR_IDENTITY_LINK_REVOKED: ({"status": "active"}, {"status": "revoked"}), + AuthorityEventType.ACTOR_IDENTITY_LINK_REACTIVATED: ({"status": "revoked"}, {"status": "active"}), + AuthorityEventType.ACTOR_PROFILE_SUSPENDED: ({"status": "active"}, {"status": "suspended"}), + AuthorityEventType.ACTOR_PROFILE_REACTIVATED: ({"status": "suspended"}, {"status": "active"}), + AuthorityEventType.PROJECT_ROLE_QUALIFICATION_CAPTURED: (None, {"status": "captured"}), + AuthorityEventType.ADMIN_ROLE_GRANT_ISSUE_DENIED: (None, None), + AuthorityEventType.LAST_ACCESS_ADMIN_OPERATION_DENIED: (None, None), + AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED: (None, {"allowed": True}), + AuthorityEventType.SENSITIVE_AUTHORIZATION_DENIED: (None, {"allowed": False}), + AuthorityEventType.AUTHORITY_INVALIDATION_REQUESTED: ({"effective": True}, {"effective": False}), + } + if event in exact: + return (before, after) == exact[event] + if event == AuthorityEventType.ACTOR_IDENTITY_LINKED: + return before is None and after in ({"status": "active", "subject_kind": "human"}, {"status": "active", "subject_kind": "service"}) + if event == AuthorityEventType.ACTOR_PROFILE_DEACTIVATED: + return before in ({"status": "active"}, {"status": "suspended"}) and after == {"status": "deactivated"} + if event == AuthorityEventType.INITIAL_ACCESS_ADMIN_BOOTSTRAPPED: + return before is None and _grant_facts(after, _ADMIN_ROLES, "active", True) and after["role"] == "access_administrator" + roles, action = { + AuthorityEventType.ADMIN_ROLE_GRANT_ISSUED: (_ADMIN_ROLES, "issued"), + AuthorityEventType.ADMIN_ROLE_GRANT_REVOKED: (_ADMIN_ROLES, "revoked"), + AuthorityEventType.PROJECT_ROLE_GRANT_ISSUED: (_PROJECT_ROLES, "issued"), + AuthorityEventType.PROJECT_ROLE_GRANT_REPLACED: (_PROJECT_ROLES, "replaced"), + AuthorityEventType.PROJECT_ROLE_GRANT_REVOKED: (_PROJECT_ROLES, "revoked"), + }[event] + if action == "issued": + return before is None and _grant_facts(after, roles, "active", True) + if action == "revoked": + return ( + _grant_facts(before, roles, "active", True) + and _grant_facts(after, roles, "revoked", False) + and (before["role"], before["scope_type"], before.get("scope_id")) + == (after["role"], after["scope_type"], after.get("scope_id")) + ) + return _grant_facts(before, roles, "active", True) and _grant_facts(after, roles, "active", True) + + class AuthorityAuditEventInput(BaseModel): """Validate one authority event without provider claims or request bodies.""" model_config = ConfigDict(extra="forbid", strict=True) - event_id: UUID = Field(default_factory=uuid4) + event_id: UUID event_type: AuthorityEventType - entity_type: SafeToken - entity_id: SafeRef36 + entity_type: str + entity_id: str actor_ref_kind: ActorReferenceKind - actor_ref: SafeRef100 + actor_ref: Annotated[str, Field(max_length=120)] request_id: UUID correlation_id: UUID target_actor_ref_kind: ActorReferenceKind | None = None - target_actor_ref: SafeRef100 | None = None - matched_grant_id: SafeRef100 | None = None - permission_id: Annotated[str, Field(max_length=120, pattern=r"^[a-z][a-z0-9_.:-]*$")] | None = None - project_id: SafeRef36 | None = None - resource_type: SafeToken | None = None - resource_id: SafeRef100 | None = None - target_ref_kind: SafeToken32 | None = None - target_ref_id: SafeRef100 | None = None - reason: str | None = Field(default=None, max_length=1000, pattern=_SAFE_REASON.pattern) - denial_code: Annotated[str, Field(max_length=80, pattern=r"^[a-z][a-z0-9_]*$")] | None = None + target_actor_ref: Annotated[str, Field(max_length=120)] | None = None + matched_grant_id: str | None = None + permission_id: str | None = None + project_id: str | None = None + resource_type: str | None = None + resource_id: str | None = None + target_ref_kind: str | None = None + target_ref_id: Annotated[str, Field(max_length=120)] | None = None + reason: str + denial_code: str | None = None idempotency_reference: UUID | None = None invalidation_cause_event_id: UUID | None = None - invalidation_target_kind: SafeToken32 | None = None - invalidation_target_ref: SafeRef100 | None = None - before_facts: dict[str, str | bool | None] | None = None - after_facts: dict[str, str | bool | None] | None = None + invalidation_target_kind: str | None = None + invalidation_target_ref: Annotated[str, Field(max_length=120)] | None = None + before_facts: dict[str, str | bool] | None = None + after_facts: dict[str, str | bool] | None = None @model_validator(mode="before") @classmethod - def reject_unknown_fields_without_echo(cls, value: object) -> object: - """Reject unknown fields before Pydantic can retain their values.""" - if isinstance(value, dict) and value.keys() - cls.model_fields.keys(): - raise TypeError("unexpected authority audit fields") - return value + def admit_privacy_safe_input(cls, value: object) -> object: + """Reject unsafe input before Pydantic can retain rejected values.""" + if not isinstance(value, Mapping) or set(value) - cls.model_fields.keys(): + raise TypeError("invalid authority audit input") + data = dict(value) + event_raw = _enum_value(data.get("event_type"), AuthorityEventType) + kind = _enum_value(data.get("actor_ref_kind"), ActorReferenceKind) + event = AuthorityEventType(event_raw) if event_raw else None + actor_ref = data.get("actor_ref") + uuid_fields = ( + "event_id", "entity_id", "request_id", "correlation_id", "matched_grant_id", + "project_id", "resource_id", "idempotency_reference", "invalidation_cause_event_id", + ) + invalid = ( + event is None + or not _registered(data.get("entity_type"), _ENTITY_TYPES) + or any(data.get(key) is not None and _uuid(data[key]) is None for key in uuid_fields) + or kind is None + or (kind == "system_principal" and actor_ref != "workstream:system:bootstrap") + or (kind != "system_principal" and _uuid(actor_ref) is None) + or data.get("permission_id") is not None and not _registered(data["permission_id"], _PERMISSIONS) + or data.get("denial_code") is not None and not _registered(data["denial_code"], _DENIAL_CODES) + or data.get("resource_type") is not None and not _registered(data["resource_type"], _RESOURCE_TYPES) + or data.get("target_ref_kind") is not None + and not _registered(data["target_ref_kind"], _UUID_TARGET_KINDS | {"permission_registry"}) + or data.get("invalidation_target_kind") is not None + and not _registered(data["invalidation_target_kind"], _UUID_TARGET_KINDS | {"permission_registry"}) + or event is not None and not _registered(data.get("reason"), _REASONS[event]) + or _facts(data.get("before_facts")) is None and data.get("before_facts") is not None + or _facts(data.get("after_facts")) is None and data.get("after_facts") is not None + ) + for prefix in ("target_ref", "invalidation_target"): + ref_kind = data.get(f"{prefix}_kind") + ref = data.get(f"{prefix}_ref" if prefix == "invalidation_target" else f"{prefix}_id") + invalid |= (ref_kind is None) != (ref is None) + invalid |= _registered(ref_kind, _UUID_TARGET_KINDS) and ref is not None and _uuid(ref) is None + invalid |= ref_kind == "permission_registry" and not _registered(ref, _PERMISSIONS) + target_kind, target_ref = data.get("target_actor_ref_kind"), data.get("target_actor_ref") + invalid |= (target_kind is None) != (target_ref is None) + invalid |= target_kind is not None and ( + _enum_value(target_kind, ActorReferenceKind) != "actor_profile" or _uuid(target_ref) is None + ) + if invalid: + raise TypeError("invalid authority audit input") + return data - @field_validator("before_facts", "after_facts", mode="before") @classmethod - def validate_facts(cls, value: object) -> object: - """Reject nested, excessive, or oversized state facts.""" - if value is None: - return None - if not isinstance(value, dict) or any(not isinstance(key, str) for key in value): - raise ValueError("invalid authority state facts") - if len(value) > 16 or any(_FACT_KEY.fullmatch(key) is None for key in value): - raise ValueError("invalid authority state facts") - if not set(value).issubset(_ALL_FACT_KEYS): - raise ValueError("authority state facts are not allowed for this event") - if any(isinstance(item, (dict, list, tuple, set)) for item in value.values()): - raise ValueError("invalid authority state facts") - if len(json.dumps(value, separators=(",", ":"), ensure_ascii=False).encode()) > 4096: - raise ValueError("authority state facts exceed size limit") - if any(not _safe_fact_value(key, item) for key, item in value.items()): - raise ValueError("invalid authority state fact value") - return value + def model_validate_json(cls, json_data: str | bytes | bytearray, **kwargs: Any) -> Self: + """Parse JSON without retaining malformed or non-object input.""" + try: + value = json.loads(json_data) + except (json.JSONDecodeError, UnicodeDecodeError, TypeError): + raise TypeError("invalid authority audit input") from None + if not isinstance(value, Mapping): + raise TypeError("invalid authority audit input") + return super().model_validate_json(json_data, **kwargs) @model_validator(mode="after") def validate_shape(self) -> Self: - """Enforce paired references and foundation event-specific fields.""" - for left, right in ( - (self.target_actor_ref_kind, self.target_actor_ref), - (self.resource_type, self.resource_id), - (self.target_ref_kind, self.target_ref_id), - (self.invalidation_target_kind, self.invalidation_target_ref), - ): - if (left is None) != (right is None): - raise ValueError("authority reference fields must be paired") - references = ((self.actor_ref_kind, self.actor_ref), (self.target_actor_ref_kind, self.target_actor_ref)) - if any(kind is not None and not _safe_actor_ref(kind, reference or "") for kind, reference in references): - raise ValueError("actor references must use canonical local identifiers") - if self.invalidation_cause_event_id == self.event_id: - raise ValueError("invalidation cannot reference its own event") + """Enforce event, reference, fact, and project-scope integrity.""" + if self.resource_id is not None and self.resource_type is None: + raise ValueError("resource ID requires resource type") + if self.entity_type in {"authorization_decision", "authority_invalidation"} and self.entity_id != str(self.event_id): + raise ValueError("decision entity ID must equal event ID") + if self.resource_type == "project" and self.resource_id is not None and self.resource_id != self.project_id: + raise ValueError("project resource must match project scope") + before, after = _facts(self.before_facts), _facts(self.after_facts) + if not _event_facts_valid(self.event_type, before, after): + raise ValueError("invalid authority event facts") + for facts in (before, after): + if facts and "scope_type" in facts: + if facts["scope_type"] == "system" and self.project_id is not None: + raise ValueError("system grant cannot carry project scope") + if facts["scope_type"] == "project" and facts.get("scope_id") != self.project_id: + raise ValueError("grant facts must match project scope") + if before and after and "scope_id" in before and before.get("scope_id") != after.get("scope_id"): + raise ValueError("replacement cannot change project scope") invalidation = self.invalidation_cause_event_id is not None or self.invalidation_target_kind - allowed_fact_keys = _EVENT_FACT_KEYS[self.event_type] - if any( - facts is not None and not set(facts).issubset(allowed_fact_keys) - for facts in (self.before_facts, self.after_facts) - ): - raise ValueError("authority state facts are not allowed for this event") if self.event_type == AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED: if self.permission_id is None or self.denial_code is not None or invalidation: raise ValueError("invalid allowed authorization evidence") elif self.event_type == AuthorityEventType.SENSITIVE_AUTHORIZATION_DENIED: - if self.permission_id is None or self.denial_code is None or invalidation: + if self.permission_id is None or self.denial_code is None or invalidation or self.idempotency_reference: raise ValueError("invalid denied authorization evidence") - if self.idempotency_reference is not None: - raise ValueError("denial cannot reference committed idempotency") elif self.event_type == AuthorityEventType.AUTHORITY_INVALIDATION_REQUESTED: - if self.invalidation_cause_event_id is None or self.invalidation_target_kind is None: + if self.invalidation_cause_event_id is None or self.invalidation_target_kind is None or self.denial_code: raise ValueError("invalid authority invalidation evidence") - if self.denial_code is not None: - raise ValueError("invalidation cannot carry denial code") + if self.invalidation_cause_event_id == self.event_id: + raise ValueError("invalidation cannot reference its own event") return self diff --git a/backend/tests/test_alembic.py b/backend/tests/test_alembic.py index 6cfc335b..ffbac20b 100644 --- a/backend/tests/test_alembic.py +++ b/backend/tests/test_alembic.py @@ -476,6 +476,7 @@ def test_authority_audit_schema_preserves_legacy_and_guards_downgrade( }, "constraints": { "ck_audit_events_authority_privacy_bounds", + "ck_audit_events_authority_registries", "ck_audit_events_authority_tokens", "ck_audit_events_domain_shape", "ck_audit_events_fact_bounds", @@ -497,6 +498,8 @@ def test_authority_audit_schema_preserves_legacy_and_guards_downgrade( }, "functions": { "authority_facts_are_safe", + "authority_grant_facts_are_safe", + "authority_event_facts_are_safe", "reject_audit_event_mutation", "set_authority_audit_database_time", }, @@ -2246,7 +2249,8 @@ async def _authority_audit_schema(database_url: str) -> dict[str, object]: await connection.execute( text( "select proname from pg_proc where proname in " - "('authority_facts_are_safe', 'reject_audit_event_mutation', " + "('authority_facts_are_safe', 'authority_grant_facts_are_safe', " + "'authority_event_facts_are_safe', 'reject_audit_event_mutation', " "'set_authority_audit_database_time')" ) ) @@ -2283,15 +2287,16 @@ async def _insert_authority_audit_fixture(database_url: str, event_id: str): "(id, entity_type, entity_id, event_type, actor_id, actor_roles, " "claim_snapshot, auth_source, is_dev_auth, event_payload, event_domain, " "event_version, occurred_at, actor_ref_kind, request_id, correlation_id, " - "permission_id) values (:id, 'actor', :entity_id, " - "'SensitiveAuthorizationAllowed', 'workstream:system:test', '[]'::json, " + "permission_id, reason, after_facts) values (:id, " + "'authorization_decision', :id, 'SensitiveAuthorizationAllowed', " + "'workstream:system:bootstrap', '[]'::json, " "'{}'::json, 'local_authority', false, '{}'::json, 'authority', 1, " "'2000-01-01T00:00:00Z', 'system_principal', :request_id, " - ":correlation_id, 'actor.manage') returning occurred_at" + ":correlation_id, 'actor.profile.read_any', " + "'authorization_evaluation', '{\"allowed\": true}') returning occurred_at" ), { "id": event_id, - "entity_id": str(uuid4()), "request_id": str(uuid4()), "correlation_id": str(uuid4()), }, diff --git a/backend/tests/test_audit.py b/backend/tests/test_audit.py index a29d57e1..94a4ec8e 100644 --- a/backend/tests/test_audit.py +++ b/backend/tests/test_audit.py @@ -1,9 +1,11 @@ from __future__ import annotations -from pathlib import Path -from uuid import UUID, uuid4 +from collections import UserDict from collections.abc import Mapping import json +from pathlib import Path +from types import MappingProxyType +from uuid import UUID, uuid4 from alembic import command from alembic.config import Config @@ -96,15 +98,35 @@ async def audit_factory(audit_database_env: str): def _authority_input(event_type: AuthorityEventType, **overrides) -> AuthorityAuditEventInput: + event_id = overrides.pop("event_id", uuid4()) + defaults = { + AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED: { + "entity_type": "authorization_decision", + "reason": "authorization_evaluation", + "after_facts": {"allowed": True}, + }, + AuthorityEventType.SENSITIVE_AUTHORIZATION_DENIED: { + "entity_type": "authorization_decision", + "reason": "authorization_evaluation", + "after_facts": {"allowed": False}, + }, + AuthorityEventType.AUTHORITY_INVALIDATION_REQUESTED: { + "entity_type": "authority_invalidation", + "reason": "authority_state_changed", + "before_facts": {"effective": True}, + "after_facts": {"effective": False}, + }, + }[event_type] values = { + "event_id": event_id, "event_type": event_type, - "entity_type": "actor", - "entity_id": str(uuid4()), + "entity_id": str(event_id), "actor_ref_kind": ActorReferenceKind.SYSTEM_PRINCIPAL, - "actor_ref": "workstream:system:test", + "actor_ref": "workstream:system:bootstrap", "request_id": uuid4(), "correlation_id": uuid4(), - "permission_id": "actor.manage", + "permission_id": "actor.profile.read_any", + **defaults, } values.update(overrides) return AuthorityAuditEventInput(**values) @@ -116,43 +138,54 @@ def test_authority_input_rejects_unbounded_or_inconsistent_evidence() -> None: mode="json" ) | {"raw_token": secret} constructors = ( - lambda: AuthorityAuditEventInput(**payload), - lambda: AuthorityAuditEventInput.model_validate(payload), - lambda: AuthorityAuditEventInput.model_validate_json(json.dumps(payload)), + lambda value: AuthorityAuditEventInput(**value), + AuthorityAuditEventInput.model_validate, ) + mappings = (payload, UserDict(payload), MappingProxyType(payload)) for constructor in constructors: - with pytest.raises(TypeError, match="unexpected authority audit fields") as caught: - constructor() - _assert_value_not_retained(caught.value, secret) - for patch in ( - {"actor_ref_kind": ActorReferenceKind.LEGACY_ACTOR, "actor_ref": "provider@example.com"}, - {"before_facts": {"status": "eyJ.raw.token"}}, - {"before_facts": {"decision_code": "https://issuer.example/private"}}, - {"reason": "See https://issuer.example/private"}, + for mapping in mappings: + with pytest.raises(TypeError, match="invalid authority audit input") as caught: + constructor(mapping) + _assert_value_not_retained(caught.value, secret) + with pytest.raises(TypeError, match="invalid authority audit input") as caught: + AuthorityAuditEventInput.model_validate_json(json.dumps(payload)) + _assert_value_not_retained(caught.value, secret) + + safe = _authority_input(AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED).model_dump( + mode="json" + ) + assert AuthorityAuditEventInput.model_validate_json(json.dumps(safe)).model_dump(mode="json") == safe + for raw in ("{secret-bearer-value", '"secret-bearer-value"', '["secret-bearer-value"]'): + for document in (raw, raw.encode(), bytearray(raw.encode())): + with pytest.raises(TypeError, match="invalid authority audit input") as caught: + AuthorityAuditEventInput.model_validate_json(document) + _assert_value_not_retained(caught.value, secret) + + for patch, forbidden in ( + ( + {"actor_ref_kind": ActorReferenceKind.LEGACY_ACTOR, "actor_ref": "provider@example.com"}, + "provider@example.com", + ), + ({"after_facts": {"allowed": "secret-bearer-value"}}, secret), + ({"after_facts": {"allowed": {"secret": secret}}}, secret), + ( + {"after_facts": {"decision_code": "https://issuer.example/private"}}, + "https://issuer.example/private", + ), + ({"reason": "secret-bearer-value"}, secret), + ({"entity_type": "secret-bearer-value"}, secret), + ({"permission_id": [secret]}, secret), ): - with pytest.raises(ValidationError): - _authority_input(AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED, **patch) - with pytest.raises(ValidationError, match="reference fields must be paired"): - _authority_input( - AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED, - resource_type="project", - ) - with pytest.raises(ValidationError, match="state facts"): + candidate = safe | patch + with pytest.raises(TypeError, match="invalid authority audit input") as caught: + AuthorityAuditEventInput.model_validate(candidate) + _assert_value_not_retained(caught.value, forbidden) + with pytest.raises(ValidationError, match="resource ID requires"): _authority_input( AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED, - before_facts={"nested": {"unsafe": True}}, + resource_id=str(uuid4()), ) - with pytest.raises(ValidationError, match="exceed size limit"): - _authority_input( - AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED, - before_facts={"status": "x" * 4096}, - ) - with pytest.raises(ValidationError, match="not allowed for this event"): - _authority_input( - AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED, - before_facts={"policy_body": "must-never-enter-evidence"}, - ) - with pytest.raises(ValidationError, match="denial cannot reference"): + with pytest.raises(ValidationError, match="invalid denied authorization"): _authority_input( AuthorityEventType.SENSITIVE_AUTHORIZATION_DENIED, denial_code="actor_suspended", @@ -167,33 +200,85 @@ def test_authority_input_rejects_unbounded_or_inconsistent_evidence() -> None: event_id=event_id, permission_id=None, invalidation_cause_event_id=event_id, - invalidation_target_kind="actor", + invalidation_target_kind="actor_profile", invalidation_target_ref=str(uuid4()), ) +def test_authority_input_enforces_grant_scope_matrix() -> None: + """Grant evidence cannot contradict role, project, or replacement scope.""" + source = _authority_input(AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED).model_dump() + event_id = uuid4() + project_id = str(uuid4()) + source.update( + event_id=event_id, + event_type=AuthorityEventType.ADMIN_ROLE_GRANT_ISSUED, + entity_type="admin_role_grant", + entity_id=str(uuid4()), + permission_id=None, + reason="authority_assignment", + project_id=project_id, + after_facts={ + "status": "active", + "role": "project_manager", + "scope_type": "project", + "scope_id": project_id, + "effective": True, + }, + ) + assert AuthorityAuditEventInput.model_validate(source).project_id == project_id + + for patch in ( + {"after_facts": source["after_facts"] | {"role": "access_administrator"}}, + {"after_facts": source["after_facts"] | {"scope_id": str(uuid4())}}, + {"project_id": None}, + ): + with pytest.raises(ValidationError, match="facts|scope"): + AuthorityAuditEventInput.model_validate(source | patch) + + replacement = source | { + "event_type": AuthorityEventType.PROJECT_ROLE_GRANT_REPLACED, + "entity_type": "project_role_grant", + "reason": "authority_replacement", + "before_facts": { + "status": "active", + "role": "submitter", + "scope_type": "project", + "scope_id": project_id, + "effective": True, + }, + "after_facts": { + "status": "active", + "role": "reviewer", + "scope_type": "project", + "scope_id": str(uuid4()), + "effective": True, + }, + } + with pytest.raises(ValidationError, match="scope"): + AuthorityAuditEventInput.model_validate(replacement) + + async def test_authority_writer_persists_typed_privacy_neutral_events(audit_factory) -> None: future_idempotency = uuid4() + project_id = str(uuid4()) allowed = _authority_input( AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED, - project_id=str(uuid4()), + project_id=project_id, resource_type="project", - resource_id=str(uuid4()), - before_facts={"status": "active"}, + resource_id=project_id, ) denied = _authority_input( AuthorityEventType.SENSITIVE_AUTHORIZATION_DENIED, - denial_code="permission_denied", - reason="Local permission was not effective.", + denial_code="permission_not_granted", ) invalidation = _authority_input( AuthorityEventType.AUTHORITY_INVALIDATION_REQUESTED, permission_id=None, invalidation_cause_event_id=allowed.event_id, - invalidation_target_kind="actor", - invalidation_target_ref="legacy:target", + invalidation_target_kind="actor_profile", + invalidation_target_ref=str(uuid4()), idempotency_reference=future_idempotency, - after_facts={"status": "revoked"}, ) async with audit_factory() as session: @@ -268,7 +353,7 @@ async def test_authority_invalidation_requires_an_existing_authority_cause(audit AuthorityEventType.AUTHORITY_INVALIDATION_REQUESTED, permission_id=None, invalidation_cause_event_id=uuid4(), - invalidation_target_kind="actor", + invalidation_target_kind="actor_profile", invalidation_target_ref=str(uuid4()), ) async with audit_factory() as session: @@ -366,7 +451,7 @@ async def test_database_rejects_malformed_and_mutated_audit_rows(audit_factory) ) ).all() assert {(row.id, row.reason, row.event_domain) for row in rows} == { - (str(value.event_id), None, "authority"), + (str(value.event_id), "authorization_evaluation", "authority"), (legacy_id, None, "legacy_lifecycle"), } @@ -386,42 +471,95 @@ async def test_database_rejects_malformed_and_mutated_audit_rows(audit_factory) "insert into audit_events " "(id, entity_type, entity_id, event_type, actor_id, actor_roles, claim_snapshot, " "auth_source, is_dev_auth, event_payload, event_domain, event_version, actor_ref_kind, " - "request_id, correlation_id, permission_id, before_facts) values " - "(:id, 'actor', :entity_id, 'SensitiveAuthorizationAllowed', :actor_id, '[]', '{}', " + "request_id, correlation_id, permission_id, reason, after_facts) values " + "(:id, :entity_type, :id, 'SensitiveAuthorizationAllowed', :actor_id, '[]', '{}', " "'local_authority', false, '{}', 'authority', 1, :actor_kind, :request_id, " - ":correlation_id, 'actor.manage', cast(:facts as json))" + ":correlation_id, 'actor.profile.read_any', :reason, cast(:facts as json))" ) - for actor_kind, actor_id, facts in ( - ("legacy_actor", "provider@example.com", "{}"), - ( - "system_principal", - "workstream:system:test", - '{"decision_code":"https://issuer.test"}', - ), - ("system_principal", "workstream:system:test", '{"status":"eyJ.raw.token"}'), + for patch in ( + {"actor_kind": "legacy_actor", "actor_id": "provider@example.com"}, + {"facts": '{"decision_code":"https://issuer.test"}'}, + {"facts": '{"allowed":"secret-bearer-value"}'}, + {"entity_type": "provider@example.com"}, + {"reason": "secret-bearer-value"}, ): with pytest.raises(IntegrityError): + values = { + "id": str(uuid4()), + "actor_id": "workstream:system:bootstrap", + "actor_kind": "system_principal", + "request_id": str(uuid4()), + "correlation_id": str(uuid4()), + "entity_type": "authorization_decision", + "reason": "authorization_evaluation", + "facts": '{"allowed":true}', + } + values.update(patch) await session.execute( insert, + values, + ) + await session.rollback() + grant_insert = text( + "insert into audit_events " + "(id, entity_type, entity_id, event_type, actor_id, actor_roles, claim_snapshot, " + "auth_source, is_dev_auth, event_payload, event_domain, event_version, actor_ref_kind, " + "request_id, correlation_id, project_id, reason, after_facts) values " + "(:id, 'admin_role_grant', :entity_id, 'AdminRoleGrantIssued', " + "'workstream:system:bootstrap', '[]', '{}', 'local_authority', false, '{}', " + "'authority', 1, 'system_principal', :request_id, :correlation_id, :project_id, " + "'authority_assignment', cast(:facts as json))" + ) + project_id = str(uuid4()) + + def grant_values(facts: str, *, scope_project_id: str | None = project_id) -> dict: + return { + "id": str(uuid4()), + "entity_id": str(uuid4()), + "request_id": str(uuid4()), + "correlation_id": str(uuid4()), + "project_id": scope_project_id, + "facts": facts, + } + + valid_facts = json.dumps( + { + "status": "active", + "role": "project_manager", + "scope_type": "project", + "scope_id": project_id, + "effective": True, + } + ) + await session.execute(grant_insert, grant_values(valid_facts)) + await session.commit() + for values in ( + grant_values(valid_facts.replace(project_id, str(uuid4()))), + grant_values(valid_facts.replace("project_manager", "access_administrator")), + grant_values( + json.dumps( { - "id": str(uuid4()), - "entity_id": str(uuid4()), - "actor_id": actor_id, - "actor_kind": actor_kind, - "request_id": str(uuid4()), - "correlation_id": str(uuid4()), - "facts": facts, - }, + "status": "active", + "role": "project_manager", + "scope_type": "system", + "effective": True, + } ) + ), + ): + with pytest.raises(IntegrityError): + await session.execute(grant_insert, values) await session.rollback() invalidation_insert = text( "insert into audit_events (id, entity_type, entity_id, event_type, actor_id, " "actor_roles, claim_snapshot, auth_source, is_dev_auth, event_payload, event_domain, " "event_version, actor_ref_kind, request_id, correlation_id, invalidation_cause_event_id, " - "invalidation_target_kind, invalidation_target_ref) values (:id, 'actor', :entity_id, " - "'AuthorityInvalidationRequested', 'workstream:system:test', '[]', '{}', " + "invalidation_target_kind, invalidation_target_ref, reason, before_facts, after_facts) " + "values (:id, 'authority_invalidation', :id, " + "'AuthorityInvalidationRequested', 'workstream:system:bootstrap', '[]', '{}', " "'local_authority', false, '{}', 'authority', 1, 'system_principal', :request_id, " - ":correlation_id, :cause_id, 'actor', :target_ref)" + ":correlation_id, :cause_id, 'actor_profile', :target_ref, " + "'authority_state_changed', '{\"effective\": true}', '{\"effective\": false}')" ) for cause_id in (str(uuid4()), legacy_id): with pytest.raises(IntegrityError): @@ -429,7 +567,6 @@ async def test_database_rejects_malformed_and_mutated_audit_rows(audit_factory) invalidation_insert, { "id": str(uuid4()), - "entity_id": str(uuid4()), "request_id": str(uuid4()), "correlation_id": str(uuid4()), "cause_id": cause_id, From bd0bde891496771f6837c2445ef5d97642fa3f0b Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 13:21:27 +0100 Subject: [PATCH 15/22] fix(auth): close authority audit custody gaps --- .agent-loop/LOOP_STATE.md | 11 +- .agent-loop/REVIEW_LOG.md | 8 + .agent-loop/WORK_QUEUE.md | 2 +- .../CHUNK_MAP.md | 10 +- .../STATUS.md | 14 +- ...001-05A-shared-audit-authority-evidence.md | 24 +- .../versions/0018_authority_audit_evidence.py | 13 +- backend/app/modules/audit/schemas.py | 35 +- backend/app/modules/audit/service.py | 1 + backend/tests/test_audit.py | 435 +++++++++++++++++- 10 files changed, 502 insertions(+), 51 deletions(-) diff --git a/.agent-loop/LOOP_STATE.md b/.agent-loop/LOOP_STATE.md index 8824f0bf..36b00488 100644 --- a/.agent-loop/LOOP_STATE.md +++ b/.agent-loop/LOOP_STATE.md @@ -16,12 +16,11 @@ - Prior `WS-AUTH-001-01` reviewed implementation SHA: `be0b836` - Prior `WS-AUTH-001-01` final merged branch head: `b5217e1` - Latest integrated `main` merge commit: `97cd0f5` -- Current gate: AUTH-05A bounded runtime repair and focused evidence after the - 1000-line circuit-breaker amendment passed at `611abfc`. -- Size checkpoint: AUTH-05A has a reviewer-required 500-line inspection and - 1000-line production hard stop after the closed typed/SQL matrix dry run; - security SQL may not be packed to satisfy it. - AUTH-05B retains its separate 350/500 boundary. +- Current gate: AUTH-05A runtime repair and focused evidence under the + human-approved semantic chunk boundary. +- Scope checkpoint: allowed files, non-goals, acceptance criteria, behavior + evidence, and required review bound AUTH-05A; there is no numeric line cap. + AUTH-05B remains a separate inactive chunk. - Next chunk: AUTH-05B remains inactive until AUTH-05A merges, post-merge memory is reconciled, and the user gives a separate explicit start. AUTH-06 remains inactive. diff --git a/.agent-loop/REVIEW_LOG.md b/.agent-loop/REVIEW_LOG.md index 47b957d1..805ad098 100644 --- a/.agent-loop/REVIEW_LOG.md +++ b/.agent-loop/REVIEW_LOG.md @@ -1,5 +1,13 @@ # Review Log +## 2026-07-14 - WS-AUTH-001-05A Semantic Scope Amendment + +The human replaced AUTH-05A's numeric production-line ceiling with the approved +chunk contract as the controlling boundary. Allowed files, explicit non-goals, +acceptance criteria, behavior evidence, and required internal review now govern +scope. Readable SQL remains mandatory; AUTH-05B, route activation, permission +evaluation, and grant lifecycle implementation remain inactive. + ## 2026-07-14 - WS-AUTH-001-05A Circuit Breaker Amendment Passed The human-approved AUTH-05A-only 500-line inspection and 1000-line hard stop diff --git a/.agent-loop/WORK_QUEUE.md b/.agent-loop/WORK_QUEUE.md index 38d6552f..3f45a464 100644 --- a/.agent-loop/WORK_QUEUE.md +++ b/.agent-loop/WORK_QUEUE.md @@ -4,7 +4,7 @@ | Chunk | Title | Risk | Status | |---|---|---:|---| -| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | 1000-line amendment passed at `611abfc`; bounded runtime repair/evidence | +| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | Human-approved semantic chunk boundary; runtime repair/evidence | ## Planned Next diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md index f7a2a0ad..efd18cba 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/CHUNK_MAP.md @@ -19,7 +19,7 @@ stopped. | `WS-AUTH-001-04A` | Request And Error Context | L1 | Merged through PR #111 as `90c9a28` | | `WS-AUTH-001-04B` | PostgreSQL Rate Controls | L1 | Merged through PR #113 as `05a63c8` | | `WS-AUTH-001-05` | Authority Evidence And Idempotency Foundation | L1 | Split before implementation into 05A and 05B | -| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | 1000-line amendment passed at `611abfc`; runtime repair/evidence | +| `WS-AUTH-001-05A` | Shared Audit Ownership And Append-Only Authority Evidence | L1 | Semantic chunk boundary approved; runtime repair/evidence | | `WS-AUTH-001-05B` | Authority Idempotency And Invalidation Foundation | L1 | Inactive pending 05A merge/memory and separate explicit start | | `WS-AUTH-001-06` | Canonical Actor Profile And Identity Link | L1 | Proposed | | `WS-AUTH-001-07` | Authorization Kernel And Permission Registry | L1 | Proposed | @@ -97,8 +97,8 @@ merged as `05a63c8` after Backend, Agent Gates, CodeRabbit, and explicit human approval passed. AUTH-04B post-merge memory then merged through PR #114 as `97cd0f5`, and the user explicitly started AUTH-05. Required plan review rejected the combined contract before runtime edits and required 05A/05B. -The first 05A implementation review proved the original 500-line ceiling +The first 05A implementation review proved the original numeric ceiling incompatible with readable typed/database privacy parity. Repaired 05A contract -review passed at `7cc6058`; the 1000-line circuit-breaker amendment passed at -`611abfc`, and bounded runtime repair is active. 05B remains inactive. Do not -start AUTH-06 or POL-002-04. +review passed at `7cc6058`; the user subsequently replaced the line cap with +the semantic AUTH-05A boundary. Runtime repair is active. 05B remains inactive. +Do not start AUTH-06 or POL-002-04. diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md index fc3d57a9..3833bb5d 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/STATUS.md @@ -49,10 +49,10 @@ audit/idempotency contract before runtime edits and required children 05A and 05B. The first repaired AUTH-05A contract passed at `7a9023b`, but exact-SHA implementation review demonstrated that closed privacy registries and readable typed/SQL parity require a larger readable contract. The repaired contract -passed at `7cc6058`, but its 650-line hard stop halted the first exact-registry -dry run at 689 lines before database parity was complete. A human-approved -1000-line ceiling amendment passed at `611abfc`; bounded runtime repair and -focused evidence are active. +passed at `7cc6058`, but numeric hard stops repeatedly interrupted the atomic +typed/database parity boundary. The human replaced the line cap with AUTH-05A's +semantic scope, acceptance criteria, behavior evidence, and review gates; +runtime repair and focused evidence are active. ## Active planning chunk @@ -79,7 +79,7 @@ None. | `WS-AUTH-001-04A` | Merged | `codex/ws-auth-001-04-request-api-controls` | #111 | Merged as `90c9a28`; production review `cdcaf77`; final branch head `36c4aa5`. | | `WS-AUTH-001-04B` | Merged | `codex/ws-auth-001-04b-postgres-rate-controls` | #113 | Merged as `05a63c8`; production review `67484b5`; final branch head `94fb2fe`. | | `WS-AUTH-001-05` | Split | `codex/ws-auth-001-05-authority-evidence` | - | Parent split before implementation into 05A and 05B. | -| `WS-AUTH-001-05A` | Runtime repair/evidence | `codex/ws-auth-001-05-authority-evidence` | - | Human-approved 1000-line ceiling passed at `611abfc`; bounded repair active. | +| `WS-AUTH-001-05A` | Runtime repair/evidence | `codex/ws-auth-001-05-authority-evidence` | - | Human-approved semantic chunk boundary; repair active. | | `WS-AUTH-001-05B` | Inactive | - | - | Idempotency and invalidation; requires 05A merge/memory and separate explicit start. | | `WS-AUTH-001-06` | Proposed | - | - | Canonical actor profile and identity link. | | `WS-AUTH-001-07` | Proposed | - | - | Authorization kernel and permissions. | @@ -95,8 +95,8 @@ None. ## Blockers -AUTH-05A has no preimplementation blocker after the human-approved 1000-line -circuit-breaker amendment passed at `611abfc`. The combined AUTH-05 contract was +AUTH-05A has no preimplementation blocker under its human-approved semantic +chunk boundary. The combined AUTH-05 contract was rejected before code changes because migration `0017` is already owned by AUTH-04B and the shared-audit plus idempotency scope was not reviewable as one L1 change. Repaired AUTH-05A owns migration `0018`; AUTH-05B later owns diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md index fa67a12e..9f7ec94e 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05A-shared-audit-authority-evidence.md @@ -2,9 +2,10 @@ ## Status -Bounded runtime repair and focused evidence. Repaired contract review passed on -`7cc6058`; the readable exact-registry dry run stopped at the prior ceiling, and -the human-approved 1000-line amendment passed at `611abfc`. +Runtime repair and focused evidence. Repaired contract review passed on +`7cc6058`; the earlier numeric amendments passed at `611abfc`, and the human +subsequently replaced the line ceiling with this chunk's semantic scope, +acceptance criteria, behavior evidence, and required-review gates. ## Parent initiative @@ -19,15 +20,14 @@ event and existing task/checker behavior. ## Risk and circuit breaker - Risk: L1 / SLA P1. -- Inspect scope at 500 changed non-comment production lines. -- Hard stop at 1000 changed non-comment production lines, counting - `backend/app/**` plus migration code; tests and evidence do not justify - exceeding the production limit. +- Scope is bounded by this chunk's allowed files, explicit non-goals, acceptance + criteria, and AUTH-05A ownership boundary rather than a production-line cap. +- Stop and replan if implementation crosses into AUTH-05B, authorization + evaluation, grant lifecycle, route activation, or another subsystem boundary. - Security SQL must remain reviewable as named functions and multiline logical - clauses. Long-line packing does not reduce the circuit-breaker numerator and - is prohibited. -- This one-time ceiling repair is limited to the reviewer-required privacy and - causation controls; it does not reactivate AUTH-05B scope. + clauses; line packing is prohibited. +- The scope amendment is limited to AUTH-05A's required privacy, custody, + registry, causation, and parity controls; it does not reactivate AUTH-05B. - A readable typed-registry dry run reached 689 changed lines before the SQL matrix was complete. The draft was discarded. Typed and database enforcement are one atomic security boundary, so they are not split across mergeable PRs. @@ -356,8 +356,6 @@ python3 scripts/check_internal_review_evidence.py changed_production_lines=$(git diff --unified=0 origin/main...HEAD -- backend/app backend/alembic | \ awk '/^\+\+\+|^---/{next} /^[+-]/{line=substr($0,2); if (line !~ /^[[:space:]]*($|#)/) n++} END{print n+0}') printf 'AUTH-05A changed production lines: %s\n' "$changed_production_lines" -if test "$changed_production_lines" -gt 500; then printf 'AUTH-05A inspection required\n'; fi -test "$changed_production_lines" -le 1000 awk 'length($0) > 120 { print FNR ":" $0; bad=1 } END { exit bad }' \ backend/alembic/versions/0018_authority_audit_evidence.py if git diff --unified=0 origin/main...HEAD -- backend/tests | \ diff --git a/backend/alembic/versions/0018_authority_audit_evidence.py b/backend/alembic/versions/0018_authority_audit_evidence.py index aa208506..02700d4f 100644 --- a/backend/alembic/versions/0018_authority_audit_evidence.py +++ b/backend/alembic/versions/0018_authority_audit_evidence.py @@ -121,7 +121,7 @@ def upgrade() -> None: create function authority_facts_are_safe(facts json) returns boolean language sql immutable strict as $$ select json_typeof(facts) = 'object' - and (select count(*) from json_each(facts)) <= 8 + and (select count(*) = count(distinct key) and count(*) <= 8 from json_each(facts)) and not exists ( select 1 from json_each(facts) item where item.key not in ( @@ -156,7 +156,7 @@ def upgrade() -> None: create function authority_grant_facts_are_safe( facts json, roles text[], expected_status text, expected_effective boolean, envelope_project_id text - ) returns boolean language sql immutable strict as $$ + ) returns boolean language sql immutable as $$ select authority_facts_are_safe(facts) and facts->>'role' = any(roles) and facts->>'status' = expected_status @@ -378,7 +378,7 @@ def upgrade() -> None: "audit_events", f""" event_domain <> 'authority' or ( - entity_type in ({entity_tokens}) and entity_id ~ '{uuid_pattern}' + id ~ '{uuid_pattern}' and entity_type in ({entity_tokens}) and entity_id ~ '{uuid_pattern}' and ( (actor_ref_kind in ('legacy_actor', 'actor_profile') and actor_id ~ '{uuid_pattern}') or (actor_ref_kind = 'system_principal' and actor_id = 'workstream:system:bootstrap') @@ -446,7 +446,12 @@ def upgrade() -> None: event_domain <> 'authority' or event_type not in ( 'SensitiveAuthorizationAllowed', 'SensitiveAuthorizationDenied', - 'AuthorityInvalidationRequested' + 'AuthorityInvalidationRequested', 'AdminRoleGrantIssueDenied', + 'LastAccessAdministratorOperationDenied' + ) + or ( + event_type in ('AdminRoleGrantIssueDenied', 'LastAccessAdministratorOperationDenied') + and denial_code is not null ) or ( event_type = 'SensitiveAuthorizationAllowed' and permission_id is not null diff --git a/backend/app/modules/audit/schemas.py b/backend/app/modules/audit/schemas.py index 142fc4af..41a6e5c3 100644 --- a/backend/app/modules/audit/schemas.py +++ b/backend/app/modules/audit/schemas.py @@ -138,6 +138,13 @@ def _registered(value: object, values: frozenset[str] | set[str]) -> bool: return isinstance(value, str) and value in values +def _unique_object(pairs: list[tuple[str, object]]) -> dict[str, object]: + value = dict(pairs) + if len(value) != len(pairs): + raise ValueError + return value + + def _facts(value: object) -> dict[str, object] | None: if value is None: return None @@ -250,8 +257,18 @@ class AuthorityAuditEventInput(BaseModel): @classmethod def admit_privacy_safe_input(cls, value: object) -> object: """Reject unsafe input before Pydantic can retain rejected values.""" - if not isinstance(value, Mapping) or set(value) - cls.model_fields.keys(): + try: + admitted = cls._inspect_privacy_safe_input(value) + except Exception: # noqa: BLE001 - hostile Mapping methods are untrusted input + admitted = None + if admitted is None: raise TypeError("invalid authority audit input") + return admitted + + @classmethod + def _inspect_privacy_safe_input(cls, value: object) -> dict | None: + if not isinstance(value, Mapping) or set(value) - cls.model_fields.keys(): + return None data = dict(value) event_raw = _enum_value(data.get("event_type"), AuthorityEventType) kind = _enum_value(data.get("actor_ref_kind"), ActorReferenceKind) @@ -290,17 +307,15 @@ def admit_privacy_safe_input(cls, value: object) -> object: invalid |= target_kind is not None and ( _enum_value(target_kind, ActorReferenceKind) != "actor_profile" or _uuid(target_ref) is None ) - if invalid: - raise TypeError("invalid authority audit input") - return data + return None if invalid else data @classmethod def model_validate_json(cls, json_data: str | bytes | bytearray, **kwargs: Any) -> Self: """Parse JSON without retaining malformed or non-object input.""" try: - value = json.loads(json_data) - except (json.JSONDecodeError, UnicodeDecodeError, TypeError): - raise TypeError("invalid authority audit input") from None + value = json.loads(json_data, object_pairs_hook=_unique_object) + except (json.JSONDecodeError, UnicodeDecodeError, TypeError, ValueError): + value = None if not isinstance(value, Mapping): raise TypeError("invalid authority audit input") return super().model_validate_json(json_data, **kwargs) @@ -332,6 +347,12 @@ def validate_shape(self) -> Self: elif self.event_type == AuthorityEventType.SENSITIVE_AUTHORIZATION_DENIED: if self.permission_id is None or self.denial_code is None or invalidation or self.idempotency_reference: raise ValueError("invalid denied authorization evidence") + elif self.event_type in { + AuthorityEventType.ADMIN_ROLE_GRANT_ISSUE_DENIED, + AuthorityEventType.LAST_ACCESS_ADMIN_OPERATION_DENIED, + }: + if self.denial_code is None: + raise ValueError("denied authority operation requires denial code") elif self.event_type == AuthorityEventType.AUTHORITY_INVALIDATION_REQUESTED: if self.invalidation_cause_event_id is None or self.invalidation_target_kind is None or self.denial_code: raise ValueError("invalid authority invalidation evidence") diff --git a/backend/app/modules/audit/service.py b/backend/app/modules/audit/service.py index db97c023..8397555f 100644 --- a/backend/app/modules/audit/service.py +++ b/backend/app/modules/audit/service.py @@ -16,6 +16,7 @@ def __init__(self, session: AsyncSession) -> None: async def add_authority_event(self, value: AuthorityAuditEventInput) -> AuditEvent: """Persist one validated authority event without committing its transaction.""" + value = AuthorityAuditEventInput.model_validate(value.model_dump()) cause_id = value.invalidation_cause_event_id if cause_id is not None and await self._repository.get_authority_event(str(cause_id)) is None: raise ValueError("invalidation cause must be an existing authority event") diff --git a/backend/tests/test_audit.py b/backend/tests/test_audit.py index 94a4ec8e..1d9255f6 100644 --- a/backend/tests/test_audit.py +++ b/backend/tests/test_audit.py @@ -38,7 +38,8 @@ def _assert_value_not_retained(value: object, forbidden: str, seen: set[int] | N _assert_value_not_retained(item, forbidden, seen) elif isinstance(value, Mapping): for key, item in value.items(): - _assert_value_not_retained((key, item), forbidden, seen) + _assert_value_not_retained(key, forbidden, seen) + _assert_value_not_retained(item, forbidden, seen) elif isinstance(value, (list, tuple, set)): for item in value: _assert_value_not_retained(item, forbidden, seen) @@ -132,8 +133,301 @@ def _authority_input(event_type: AuthorityEventType, **overrides) -> AuthorityAu return AuthorityAuditEventInput(**values) +def _authority_event_matrix() -> list[dict]: + """Return every closed event with one valid and one fact-invalid shape.""" + project_id = str(uuid4()) + system_active = { + "status": "active", + "role": "access_administrator", + "scope_type": "system", + "effective": True, + } + project_active = { + "status": "active", + "role": "submitter", + "scope_type": "project", + "scope_id": project_id, + "effective": True, + } + + def row(event_type, entity_type, reason, before, after, invalid_after, **extra): + event_id = uuid4() + return { + "event_id": event_id, + "event_type": event_type, + "entity_type": entity_type, + "entity_id": str(event_id) + if entity_type in {"authorization_decision", "authority_invalidation"} + else str(uuid4()), + "actor_ref_kind": ActorReferenceKind.SYSTEM_PRINCIPAL, + "actor_ref": "workstream:system:bootstrap", + "request_id": uuid4(), + "correlation_id": uuid4(), + "reason": reason, + "before_facts": before, + "after_facts": after, + "invalid_patch": {"after_facts": invalid_after}, + **extra, + } + + rows = [ + row( + AuthorityEventType.ACTOR_PROFILE_PROVISIONED, + "actor_profile", + "automatic_first_access", + None, + { + "status": "active", + "subject_kind": "human", + "provisioning_method": "automatic_first_access", + }, + { + "status": "active", + "subject_kind": "service", + "provisioning_method": "automatic_first_access", + }, + ), + row( + AuthorityEventType.SERVICE_ACTOR_PROVISIONED, + "actor_profile", + "manual_service_provisioning", + None, + { + "status": "active", + "subject_kind": "service", + "provisioning_method": "manual_service_provisioning", + }, + { + "status": "active", + "subject_kind": "human", + "provisioning_method": "manual_service_provisioning", + }, + ), + row( + AuthorityEventType.ACTOR_IDENTITY_LINKED, + "actor_identity_link", + "identity_lifecycle_change", + None, + {"status": "active", "subject_kind": "human"}, + {"status": "revoked", "subject_kind": "human"}, + ), + row( + AuthorityEventType.ACTOR_IDENTITY_LINK_REVOKED, + "actor_identity_link", + "identity_lifecycle_change", + {"status": "active"}, + {"status": "revoked"}, + {"status": "active"}, + ), + row( + AuthorityEventType.ACTOR_IDENTITY_LINK_REACTIVATED, + "actor_identity_link", + "identity_lifecycle_change", + {"status": "revoked"}, + {"status": "active"}, + {"status": "suspended"}, + ), + row( + AuthorityEventType.ACTOR_PROFILE_SUSPENDED, + "actor_profile", + "security_response", + {"status": "active"}, + {"status": "suspended"}, + {"status": "deactivated"}, + ), + row( + AuthorityEventType.ACTOR_PROFILE_REACTIVATED, + "actor_profile", + "administrative_correction", + {"status": "suspended"}, + {"status": "active"}, + {"status": "deactivated"}, + ), + row( + AuthorityEventType.ACTOR_PROFILE_DEACTIVATED, + "actor_profile", + "security_response", + {"status": "active"}, + {"status": "deactivated"}, + {"status": "suspended"}, + ), + row( + AuthorityEventType.INITIAL_ACCESS_ADMIN_BOOTSTRAPPED, + "admin_role_grant", + "initial_access_bootstrap", + None, + system_active, + system_active | {"effective": False}, + ), + row( + AuthorityEventType.ADMIN_ROLE_GRANT_ISSUED, + "admin_role_grant", + "authority_assignment", + None, + system_active, + system_active | {"status": "revoked"}, + ), + row( + AuthorityEventType.ADMIN_ROLE_GRANT_REVOKED, + "admin_role_grant", + "authority_revocation", + system_active, + system_active | {"status": "revoked", "effective": False}, + system_active | {"role": "operator", "status": "revoked", "effective": False}, + ), + row( + AuthorityEventType.ADMIN_ROLE_GRANT_ISSUE_DENIED, + "admin_role_grant", + "authorization_policy_denial", + None, + None, + {"allowed": False}, + denial_code="admin_role_grant_exists", + ), + row( + AuthorityEventType.LAST_ACCESS_ADMIN_OPERATION_DENIED, + "admin_role_grant", + "authorization_policy_denial", + None, + None, + {"allowed": False}, + denial_code="last_access_administrator", + ), + row( + AuthorityEventType.PROJECT_ROLE_QUALIFICATION_CAPTURED, + "qualification_snapshot", + "qualification_evidence_captured", + None, + {"status": "captured"}, + {"status": "active"}, + project_id=project_id, + ), + row( + AuthorityEventType.PROJECT_ROLE_GRANT_ISSUED, + "project_role_grant", + "authority_assignment", + None, + project_active, + { + "status": "active", + "role": "submitter", + "scope_type": "system", + "effective": True, + }, + project_id=project_id, + ), + row( + AuthorityEventType.PROJECT_ROLE_GRANT_REPLACED, + "project_role_grant", + "authority_replacement", + project_active, + project_active | {"role": "reviewer"}, + project_active | {"scope_id": str(uuid4()), "role": "reviewer"}, + project_id=project_id, + ), + row( + AuthorityEventType.PROJECT_ROLE_GRANT_REVOKED, + "project_role_grant", + "authority_revocation", + project_active, + project_active | {"status": "revoked", "effective": False}, + project_active | {"role": "reviewer", "status": "revoked", "effective": False}, + project_id=project_id, + ), + row( + AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED, + "authorization_decision", + "authorization_evaluation", + None, + {"allowed": True}, + {"allowed": False}, + permission_id="actor.profile.read_any", + ), + row( + AuthorityEventType.SENSITIVE_AUTHORIZATION_DENIED, + "authorization_decision", + "authorization_evaluation", + None, + {"allowed": False}, + {"allowed": True}, + permission_id="actor.profile.read_any", + denial_code="actor_suspended", + ), + ] + rows[11]["invalid_patch"] = {"denial_code": None} + rows[12]["invalid_patch"] = {"denial_code": None} + rows.append( + row( + AuthorityEventType.AUTHORITY_INVALIDATION_REQUESTED, + "authority_invalidation", + "authority_state_changed", + {"effective": True}, + {"effective": False}, + {"effective": True}, + invalidation_cause_event_id=rows[0]["event_id"], + invalidation_target_kind="actor_profile", + invalidation_target_ref=str(uuid4()), + ) + ) + return rows + + +def _authority_sql_values(candidate: dict) -> dict: + """Convert one shared matrix row into direct-SQL bind values.""" + value = candidate.copy() + value.pop("invalid_patch", None) + optional = ( + "target_actor_ref_kind", + "target_actor_ref", + "matched_grant_id", + "permission_id", + "project_id", + "resource_type", + "resource_id", + "target_ref_kind", + "target_ref_id", + "denial_code", + "idempotency_reference", + "invalidation_cause_event_id", + "invalidation_target_kind", + "invalidation_target_ref", + ) + return { + **{key: value.get(key) for key in optional}, + "id": str(value["event_id"]), + "entity_type": value["entity_type"], + "entity_id": value["entity_id"], + "event_type": value["event_type"].value, + "actor_ref_kind": value["actor_ref_kind"].value, + "actor_id": value["actor_ref"], + "request_id": str(value["request_id"]), + "correlation_id": str(value["correlation_id"]), + "reason": value["reason"], + "invalidation_cause_event_id": str(value["invalidation_cause_event_id"]) + if value.get("invalidation_cause_event_id") is not None + else None, + "before_facts": json.dumps(value["before_facts"]) + if value.get("before_facts") is not None + else None, + "after_facts": json.dumps(value["after_facts"]) + if value.get("after_facts") is not None + else None, + } + + def test_authority_input_rejects_unbounded_or_inconsistent_evidence() -> None: secret = "secret-bearer-value" + + class HostileMapping(Mapping): + def __getitem__(self, key): + raise KeyError(key) + + def __iter__(self): + raise RuntimeError(secret) + + def __len__(self): + return 1 + payload = _authority_input(AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED).model_dump( mode="json" ) | {"raw_token": secret} @@ -150,12 +444,28 @@ def test_authority_input_rejects_unbounded_or_inconsistent_evidence() -> None: with pytest.raises(TypeError, match="invalid authority audit input") as caught: AuthorityAuditEventInput.model_validate_json(json.dumps(payload)) _assert_value_not_retained(caught.value, secret) + with pytest.raises(TypeError, match="invalid authority audit input") as caught: + AuthorityAuditEventInput.model_validate(HostileMapping()) + _assert_value_not_retained(caught.value, secret) safe = _authority_input(AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED).model_dump( mode="json" ) - assert AuthorityAuditEventInput.model_validate_json(json.dumps(safe)).model_dump(mode="json") == safe - for raw in ("{secret-bearer-value", '"secret-bearer-value"', '["secret-bearer-value"]'): + assert ( + AuthorityAuditEventInput.model_validate_json(json.dumps(safe)).model_dump(mode="json") + == safe + ) + duplicate = json.dumps(safe).replace('"allowed": true', '"allowed": false, "allowed": true') + for raw in ( + "{secret-bearer-value", + '"secret-bearer-value"', + '["secret-bearer-value"]', + "0", + "true", + "false", + "null", + duplicate, + ): for document in (raw, raw.encode(), bytearray(raw.encode())): with pytest.raises(TypeError, match="invalid authority audit input") as caught: AuthorityAuditEventInput.model_validate_json(document) @@ -163,7 +473,10 @@ def test_authority_input_rejects_unbounded_or_inconsistent_evidence() -> None: for patch, forbidden in ( ( - {"actor_ref_kind": ActorReferenceKind.LEGACY_ACTOR, "actor_ref": "provider@example.com"}, + { + "actor_ref_kind": ActorReferenceKind.LEGACY_ACTOR, + "actor_ref": "provider@example.com", + }, "provider@example.com", ), ({"after_facts": {"allowed": "secret-bearer-value"}}, secret), @@ -177,8 +490,12 @@ def test_authority_input_rejects_unbounded_or_inconsistent_evidence() -> None: ({"permission_id": [secret]}, secret), ): candidate = safe | patch + for constructor in constructors: + with pytest.raises(TypeError, match="invalid authority audit input") as caught: + constructor(candidate) + _assert_value_not_retained(caught.value, forbidden) with pytest.raises(TypeError, match="invalid authority audit input") as caught: - AuthorityAuditEventInput.model_validate(candidate) + AuthorityAuditEventInput.model_validate_json(json.dumps(candidate)) _assert_value_not_retained(caught.value, forbidden) with pytest.raises(ValidationError, match="resource ID requires"): _authority_input( @@ -257,6 +574,78 @@ def test_authority_input_enforces_grant_scope_matrix() -> None: } with pytest.raises(ValidationError, match="scope"): AuthorityAuditEventInput.model_validate(replacement) + with pytest.raises(ValidationError, match="project resource"): + AuthorityAuditEventInput.model_validate( + source + | { + "resource_type": "project", + "resource_id": str(uuid4()), + } + ) + + +async def test_authority_event_matrix_has_typed_direct_sql_parity(audit_factory) -> None: + """Every registered event accepts and rejects the same fact shape at both boundaries.""" + insert = text( + "insert into audit_events " + "(id, entity_type, entity_id, event_type, actor_id, actor_roles, claim_snapshot, " + "auth_source, is_dev_auth, event_payload, event_domain, event_version, actor_ref_kind, " + "request_id, correlation_id, target_actor_ref_kind, target_actor_ref, matched_grant_id, " + "permission_id, project_id, resource_type, resource_id, target_ref_kind, target_ref_id, " + "reason, denial_code, idempotency_reference, invalidation_cause_event_id, " + "invalidation_target_kind, invalidation_target_ref, before_facts, after_facts) values " + "(:id, :entity_type, :entity_id, :event_type, :actor_id, '[]', '{}', " + "'local_authority', false, '{}', 'authority', 1, :actor_ref_kind, :request_id, " + ":correlation_id, :target_actor_ref_kind, :target_actor_ref, :matched_grant_id, " + ":permission_id, :project_id, :resource_type, :resource_id, :target_ref_kind, " + ":target_ref_id, :reason, :denial_code, :idempotency_reference, " + ":invalidation_cause_event_id, :invalidation_target_kind, :invalidation_target_ref, " + "cast(:before_facts as json), cast(:after_facts as json))" + ) + cases = _authority_event_matrix() + assert {case["event_type"] for case in cases} == set(AuthorityEventType) + + async with audit_factory() as session: + for case in cases: + candidate = {key: value for key, value in case.items() if key != "invalid_patch"} + assert ( + AuthorityAuditEventInput.model_validate(candidate).event_type == case["event_type"] + ) + await session.execute(insert, _authority_sql_values(candidate)) + await session.commit() + + for case in cases: + event_id = uuid4() + candidate = { + **{key: value for key, value in case.items() if key != "invalid_patch"}, + **case["invalid_patch"], + "event_id": event_id, + } + if candidate["entity_type"] in {"authorization_decision", "authority_invalidation"}: + candidate["entity_id"] = str(event_id) + with pytest.raises(ValidationError): + AuthorityAuditEventInput.model_validate(candidate) + with pytest.raises(IntegrityError): + await session.execute(insert, _authority_sql_values(candidate)) + await session.rollback() + + +async def test_authority_service_readmits_mutated_inputs_without_retention(audit_factory) -> None: + """The service never forwards post-validation mutations to SQLAlchemy.""" + secret = "secret-bearer-value" + values = [ + _authority_input(AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED), + _authority_input(AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED), + ] + values[0].actor_ref = secret + values[1].after_facts["allowed"] = secret + + async with audit_factory() as session: + for value in values: + with pytest.raises(TypeError, match="invalid authority audit input") as caught: + await AuditService(session).add_authority_event(value) + _assert_value_not_retained(caught.value, secret) + assert await session.get(AuditEvent, str(value.event_id)) is None async def test_authority_writer_persists_typed_privacy_neutral_events(audit_factory) -> None: @@ -474,13 +863,16 @@ async def test_database_rejects_malformed_and_mutated_audit_rows(audit_factory) "request_id, correlation_id, permission_id, reason, after_facts) values " "(:id, :entity_type, :id, 'SensitiveAuthorizationAllowed', :actor_id, '[]', '{}', " "'local_authority', false, '{}', 'authority', 1, :actor_kind, :request_id, " - ":correlation_id, 'actor.profile.read_any', :reason, cast(:facts as json))" + ":correlation_id, :permission_id, :reason, cast(:facts as json))" ) for patch in ( {"actor_kind": "legacy_actor", "actor_id": "provider@example.com"}, {"facts": '{"decision_code":"https://issuer.test"}'}, {"facts": '{"allowed":"secret-bearer-value"}'}, + {"facts": '{"allowed":false,"allowed":true}'}, {"entity_type": "provider@example.com"}, + {"id": "secret-bearer-value"}, + {"permission_id": "secret-bearer-value"}, {"reason": "secret-bearer-value"}, ): with pytest.raises(IntegrityError): @@ -491,6 +883,7 @@ async def test_database_rejects_malformed_and_mutated_audit_rows(audit_factory) "request_id": str(uuid4()), "correlation_id": str(uuid4()), "entity_type": "authorization_decision", + "permission_id": "actor.profile.read_any", "reason": "authorization_evaluation", "facts": '{"allowed":true}', } @@ -504,11 +897,12 @@ async def test_database_rejects_malformed_and_mutated_audit_rows(audit_factory) "insert into audit_events " "(id, entity_type, entity_id, event_type, actor_id, actor_roles, claim_snapshot, " "auth_source, is_dev_auth, event_payload, event_domain, event_version, actor_ref_kind, " - "request_id, correlation_id, project_id, reason, after_facts) values " + "request_id, correlation_id, project_id, resource_type, resource_id, reason, " + "after_facts) values " "(:id, 'admin_role_grant', :entity_id, 'AdminRoleGrantIssued', " "'workstream:system:bootstrap', '[]', '{}', 'local_authority', false, '{}', " "'authority', 1, 'system_principal', :request_id, :correlation_id, :project_id, " - "'authority_assignment', cast(:facts as json))" + ":resource_type, :resource_id, 'authority_assignment', cast(:facts as json))" ) project_id = str(uuid4()) @@ -519,6 +913,8 @@ def grant_values(facts: str, *, scope_project_id: str | None = project_id) -> di "request_id": str(uuid4()), "correlation_id": str(uuid4()), "project_id": scope_project_id, + "resource_type": "project" if scope_project_id is not None else None, + "resource_id": scope_project_id, "facts": facts, } @@ -532,9 +928,19 @@ def grant_values(facts: str, *, scope_project_id: str | None = project_id) -> di } ) await session.execute(grant_insert, grant_values(valid_facts)) + system_facts = json.dumps( + { + "status": "active", + "role": "access_administrator", + "scope_type": "system", + "effective": True, + } + ) + await session.execute(grant_insert, grant_values(system_facts, scope_project_id=None)) await session.commit() for values in ( grant_values(valid_facts.replace(project_id, str(uuid4()))), + grant_values(valid_facts) | {"resource_id": str(uuid4())}, grant_values(valid_facts.replace("project_manager", "access_administrator")), grant_values( json.dumps( @@ -574,3 +980,16 @@ def grant_values(facts: str, *, scope_project_id: str | None = project_id) -> di }, ) await session.rollback() + self_id = str(uuid4()) + with pytest.raises(IntegrityError): + await session.execute( + invalidation_insert, + { + "id": self_id, + "request_id": str(uuid4()), + "correlation_id": str(uuid4()), + "cause_id": self_id, + "target_ref": str(uuid4()), + }, + ) + await session.rollback() From 1db2a3604b98428fd31355286be3f8dd265158f1 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 13:33:02 +0100 Subject: [PATCH 16/22] fix(auth): snapshot untrusted audit inputs --- backend/app/modules/audit/schemas.py | 15 +++++-- backend/app/modules/audit/service.py | 8 +++- backend/tests/test_audit.py | 66 ++++++++++++++++++++++++++-- 3 files changed, 81 insertions(+), 8 deletions(-) diff --git a/backend/app/modules/audit/schemas.py b/backend/app/modules/audit/schemas.py index 41a6e5c3..5882525d 100644 --- a/backend/app/modules/audit/schemas.py +++ b/backend/app/modules/audit/schemas.py @@ -267,9 +267,11 @@ def admit_privacy_safe_input(cls, value: object) -> object: @classmethod def _inspect_privacy_safe_input(cls, value: object) -> dict | None: - if not isinstance(value, Mapping) or set(value) - cls.model_fields.keys(): + if not isinstance(value, Mapping): return None data = dict(value) + if set(data) - cls.model_fields.keys(): + return None event_raw = _enum_value(data.get("event_type"), AuthorityEventType) kind = _enum_value(data.get("actor_ref_kind"), ActorReferenceKind) event = AuthorityEventType(event_raw) if event_raw else None @@ -313,12 +315,17 @@ def _inspect_privacy_safe_input(cls, value: object) -> dict | None: def model_validate_json(cls, json_data: str | bytes | bytearray, **kwargs: Any) -> Self: """Parse JSON without retaining malformed or non-object input.""" try: - value = json.loads(json_data, object_pairs_hook=_unique_object) - except (json.JSONDecodeError, UnicodeDecodeError, TypeError, ValueError): + snapshot = ( + str.encode(json_data) + if isinstance(json_data, str) + else memoryview(json_data).tobytes() + ) + value = json.loads(snapshot, object_pairs_hook=_unique_object) + except Exception: # noqa: BLE001 - hostile buffer methods are untrusted input value = None if not isinstance(value, Mapping): raise TypeError("invalid authority audit input") - return super().model_validate_json(json_data, **kwargs) + return super().model_validate_json(json.dumps(value, separators=(",", ":")), **kwargs) @model_validator(mode="after") def validate_shape(self) -> Self: diff --git a/backend/app/modules/audit/service.py b/backend/app/modules/audit/service.py index 8397555f..95b28db5 100644 --- a/backend/app/modules/audit/service.py +++ b/backend/app/modules/audit/service.py @@ -16,7 +16,13 @@ def __init__(self, session: AsyncSession) -> None: async def add_authority_event(self, value: AuthorityAuditEventInput) -> AuditEvent: """Persist one validated authority event without committing its transaction.""" - value = AuthorityAuditEventInput.model_validate(value.model_dump()) + try: + fields = dict(object.__getattribute__(value, "__dict__")) + except Exception: # noqa: BLE001 - the service boundary accepts no caller diagnostics + fields = None + if fields is None: + raise TypeError("invalid authority audit input") + value = AuthorityAuditEventInput.model_validate(fields) cause_id = value.invalidation_cause_event_id if cause_id is not None and await self._repository.get_authority_event(str(cause_id)) is None: raise ValueError("invalidation cause must be an existing authority event") diff --git a/backend/tests/test_audit.py b/backend/tests/test_audit.py index 1d9255f6..2a9d6c2c 100644 --- a/backend/tests/test_audit.py +++ b/backend/tests/test_audit.py @@ -6,6 +6,7 @@ from pathlib import Path from types import MappingProxyType from uuid import UUID, uuid4 +import warnings from alembic import command from alembic.config import Config @@ -451,11 +452,48 @@ def __len__(self): safe = _authority_input(AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED).model_dump( mode="json" ) + safe_python = _authority_input( + AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED + ).model_dump() + + class ChangingMapping(Mapping): + def __init__(self): + self.iterations = 0 + + def __getitem__(self, key): + return safe_python[key] if key in safe_python else secret + + def __iter__(self): + self.iterations += 1 + keys = ( + safe_python + if self.iterations == 1 + else safe_python | {"raw_token": secret} + ) + return iter(keys) + + def __len__(self): + return len(safe_python) + + class HostileBytearray(bytearray): + def decode(self, *args, **kwargs): + raise RuntimeError(secret) + + class LyingBytearray(bytearray): + def decode(self, *args, **kwargs): + return json.dumps(safe) + + changing = ChangingMapping() + assert AuthorityAuditEventInput.model_validate(changing).event_type == safe_python["event_type"] + assert changing.iterations == 1 assert ( AuthorityAuditEventInput.model_validate_json(json.dumps(safe)).model_dump(mode="json") == safe ) - duplicate = json.dumps(safe).replace('"allowed": true', '"allowed": false, "allowed": true') + duplicate = json.dumps(safe).replace( + '"allowed": true', + '"allowed": false, "allowed": true, "raw_token": "secret-bearer-value"', + ) for raw in ( "{secret-bearer-value", '"secret-bearer-value"', @@ -470,6 +508,10 @@ def __len__(self): with pytest.raises(TypeError, match="invalid authority audit input") as caught: AuthorityAuditEventInput.model_validate_json(document) _assert_value_not_retained(caught.value, secret) + for document in (HostileBytearray(b"{}"), LyingBytearray(duplicate.encode())): + with pytest.raises(TypeError, match="invalid authority audit input") as caught: + AuthorityAuditEventInput.model_validate_json(document) + _assert_value_not_retained(caught.value, secret) for patch, forbidden in ( ( @@ -633,18 +675,36 @@ async def test_authority_event_matrix_has_typed_direct_sql_parity(audit_factory) async def test_authority_service_readmits_mutated_inputs_without_retention(audit_factory) -> None: """The service never forwards post-validation mutations to SQLAlchemy.""" secret = "secret-bearer-value" + + class HostileFacts(Mapping): + def __getitem__(self, key): + raise KeyError(key) + + def __iter__(self): + raise RuntimeError(secret) + + def __len__(self): + return 1 + + def __repr__(self): + return f"<{secret}>" + values = [ _authority_input(AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED), _authority_input(AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED), + _authority_input(AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED), ] values[0].actor_ref = secret values[1].after_facts["allowed"] = secret + values[2].after_facts = HostileFacts() async with audit_factory() as session: for value in values: - with pytest.raises(TypeError, match="invalid authority audit input") as caught: - await AuditService(session).add_authority_event(value) + with warnings.catch_warnings(record=True) as caught_warnings: + with pytest.raises(TypeError, match="invalid authority audit input") as caught: + await AuditService(session).add_authority_event(value) _assert_value_not_retained(caught.value, secret) + _assert_value_not_retained(caught_warnings, secret) assert await session.get(AuditEvent, str(value.event_id)) is None From 78d34ca754be02b33c29306c95719815c0f22636 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 13:35:34 +0100 Subject: [PATCH 17/22] test(auth): require silent audit readmission --- backend/tests/test_audit.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/backend/tests/test_audit.py b/backend/tests/test_audit.py index 2a9d6c2c..04a68db7 100644 --- a/backend/tests/test_audit.py +++ b/backend/tests/test_audit.py @@ -704,7 +704,7 @@ def __repr__(self): with pytest.raises(TypeError, match="invalid authority audit input") as caught: await AuditService(session).add_authority_event(value) _assert_value_not_retained(caught.value, secret) - _assert_value_not_retained(caught_warnings, secret) + assert caught_warnings == [] assert await session.get(AuditEvent, str(value.event_id)) is None From 44901286b5c867a414cc39a9ccff5307bd23ad52 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 13:39:08 +0100 Subject: [PATCH 18/22] fix(auth): normalize nested audit facts --- backend/app/modules/audit/schemas.py | 20 +++++++---- backend/tests/test_audit.py | 50 ++++++++++++++++++++++++++++ 2 files changed, 64 insertions(+), 6 deletions(-) diff --git a/backend/app/modules/audit/schemas.py b/backend/app/modules/audit/schemas.py index 5882525d..62b77f0d 100644 --- a/backend/app/modules/audit/schemas.py +++ b/backend/app/modules/audit/schemas.py @@ -148,11 +148,13 @@ def _unique_object(pairs: list[tuple[str, object]]) -> dict[str, object]: def _facts(value: object) -> dict[str, object] | None: if value is None: return None - if not isinstance(value, Mapping) or len(value) > 8: - return None - if not set(value).issubset(frozenset(_FACT_VALUES) | {"effective", "allowed", "scope_id"}): + if not isinstance(value, Mapping): return None data = dict(value) + if len(data) > 8 or not set(data).issubset( + frozenset(_FACT_VALUES) | {"effective", "allowed", "scope_id"} + ): + return None for key, item in data.items(): if key in {"effective", "allowed"} and type(item) is not bool: return None @@ -276,6 +278,8 @@ def _inspect_privacy_safe_input(cls, value: object) -> dict | None: kind = _enum_value(data.get("actor_ref_kind"), ActorReferenceKind) event = AuthorityEventType(event_raw) if event_raw else None actor_ref = data.get("actor_ref") + before_facts = _facts(data.get("before_facts")) + after_facts = _facts(data.get("after_facts")) uuid_fields = ( "event_id", "entity_id", "request_id", "correlation_id", "matched_grant_id", "project_id", "resource_id", "idempotency_reference", "invalidation_cause_event_id", @@ -295,8 +299,8 @@ def _inspect_privacy_safe_input(cls, value: object) -> dict | None: or data.get("invalidation_target_kind") is not None and not _registered(data["invalidation_target_kind"], _UUID_TARGET_KINDS | {"permission_registry"}) or event is not None and not _registered(data.get("reason"), _REASONS[event]) - or _facts(data.get("before_facts")) is None and data.get("before_facts") is not None - or _facts(data.get("after_facts")) is None and data.get("after_facts") is not None + or before_facts is None and data.get("before_facts") is not None + or after_facts is None and data.get("after_facts") is not None ) for prefix in ("target_ref", "invalidation_target"): ref_kind = data.get(f"{prefix}_kind") @@ -309,7 +313,11 @@ def _inspect_privacy_safe_input(cls, value: object) -> dict | None: invalid |= target_kind is not None and ( _enum_value(target_kind, ActorReferenceKind) != "actor_profile" or _uuid(target_ref) is None ) - return None if invalid else data + if invalid: + return None + data["before_facts"] = before_facts + data["after_facts"] = after_facts + return data @classmethod def model_validate_json(cls, json_data: str | bytes | bytearray, **kwargs: Any) -> Self: diff --git a/backend/tests/test_audit.py b/backend/tests/test_audit.py index 04a68db7..d8e57d94 100644 --- a/backend/tests/test_audit.py +++ b/backend/tests/test_audit.py @@ -483,9 +483,34 @@ class LyingBytearray(bytearray): def decode(self, *args, **kwargs): return json.dumps(safe) + class ChangingFacts(Mapping): + def __init__(self): + self.iterations = 0 + + def __getitem__(self, key): + return True if key == "allowed" else secret + + def __iter__(self): + self.iterations += 1 + return iter(("allowed",) if self.iterations == 1 else ("raw_token",)) + + def __len__(self): + return 1 + + def __repr__(self): + return f"<{secret}>" + changing = ChangingMapping() assert AuthorityAuditEventInput.model_validate(changing).event_type == safe_python["event_type"] assert changing.iterations == 1 + changing_facts = ChangingFacts() + with warnings.catch_warnings(record=True) as caught_warnings: + normalized = AuthorityAuditEventInput.model_validate( + safe_python | {"after_facts": changing_facts} + ) + assert normalized.after_facts == {"allowed": True} + assert changing_facts.iterations == 1 + assert caught_warnings == [] assert ( AuthorityAuditEventInput.model_validate_json(json.dumps(safe)).model_dump(mode="json") == safe @@ -689,6 +714,23 @@ def __len__(self): def __repr__(self): return f"<{secret}>" + class ChangingFacts(Mapping): + def __init__(self): + self.iterations = 0 + + def __getitem__(self, key): + return True if key == "allowed" else secret + + def __iter__(self): + self.iterations += 1 + return iter(("allowed",) if self.iterations == 1 else ("raw_token",)) + + def __len__(self): + return 1 + + def __repr__(self): + return f"<{secret}>" + values = [ _authority_input(AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED), _authority_input(AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED), @@ -697,6 +739,9 @@ def __repr__(self): values[0].actor_ref = secret values[1].after_facts["allowed"] = secret values[2].after_facts = HostileFacts() + valid_hostile_repr = _authority_input(AuthorityEventType.SENSITIVE_AUTHORIZATION_ALLOWED) + changing_facts = ChangingFacts() + valid_hostile_repr.after_facts = changing_facts async with audit_factory() as session: for value in values: @@ -706,6 +751,11 @@ def __repr__(self): _assert_value_not_retained(caught.value, secret) assert caught_warnings == [] assert await session.get(AuditEvent, str(value.event_id)) is None + with warnings.catch_warnings(record=True) as caught_warnings: + stored = await AuditService(session).add_authority_event(valid_hostile_repr) + assert stored.after_facts == {"allowed": True} + assert changing_facts.iterations == 1 + assert caught_warnings == [] async def test_authority_writer_persists_typed_privacy_neutral_events(audit_factory) -> None: From 3536e399d4d3712c8c0063c77e32d83358783474 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 13:46:33 +0100 Subject: [PATCH 19/22] docs(auth): record AUTH-05A review evidence --- ...S-AUTH-001-05A-internal-review-evidence.md | 58 ++++++++ .../WS-AUTH-001-05A-pr-trust-bundle.md | 133 ++++++++++++++++++ 2 files changed, 191 insertions(+) create mode 100644 .agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-internal-review-evidence.md create mode 100644 .agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-pr-trust-bundle.md diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-internal-review-evidence.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-internal-review-evidence.md new file mode 100644 index 00000000..89bb2107 --- /dev/null +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-internal-review-evidence.md @@ -0,0 +1,58 @@ +# WS-AUTH-001-05A Internal Review Evidence + +Reviewed code SHA: `44901286b5c867a414cc39a9ccff5307bd23ad52` +Reviewed at: `2026-07-14T12:45:18Z` +Reviewer run IDs: `/root/auth04b_final_eng`, `/root/auth04b_final_qa`, `/root/auth04b_final_security` + +## Deterministic Evidence + +- Exact migration upgrade/downgrade proof: 1 passed in 52.49 seconds. +- Audit behavior suite: 10 passed in 29.84 seconds. +- Audit plus shared-repository delegation coverage: 11 passed in 98.42 + seconds; `app.modules.audit` reached 94.55 percent against the 90 percent + gate. +- Task compatibility regressions for manual checker bypass and missing lock + audit: 2 passed in 154.57 seconds. Shared repository delegation also passed + in the coverage run. +- Ruff passed for `backend/app`, `backend/tests`, and `backend/alembic`. +- Docstring coverage passed at 95.1 percent overall. +- Test-delta analysis reported three changed test files and no weakening; + deleted assertion, raises, skip, and xfail scans passed. +- Migration lines remained within 120 characters. `git diff --check`, stale + Workstream/auth/artifact scans, and changed Markdown link checks passed. +- No dependency, workflow, coverage threshold, exclusion, or package-script + change was made. +- The multi-hour repository suite was not repeated locally. GitHub Backend CI + remains responsible for the full suite and repository-wide 78 percent floor; + the latest merged main evidence was 82.15 percent. + +## Reviewer Results + +| Reviewer | Result | Blocking findings | Notes | +|---|---|---|---| +| senior engineering | PASS | none | Final normalized Mapping custody and service snapshot reviewed. | +| qa/test | PASS | none | Full event matrix, warning assertion, and behavior regressions reviewed. | +| security/auth | PASS | none | Privacy non-retention, registries, causes, and append-only controls reviewed. | +| product/ops | PASS | none | Authority evidence semantics pass; no lifecycle activation entered scope. | +| architecture | PASS | none | Shared writer and caller-owned transaction boundaries pass. | +| ci integrity | PASS | none | CI, thresholds, dependencies, and exclusions were unchanged. | +| docs | PASS | none | Architecture, operations, and durable loop state match the implementation. | +| reuse/dedup | PASS | none | Task compatibility methods delegate to the shared audit repository. | +| test delta | PASS | none | No assertion, raise, skip, xfail, or behavior guard was weakened. | + +## Findings Resolved + +Review cycles repaired valid findings covering system-scope SQL `NULL` +semantics, canonical UUID event IDs, duplicate JSON keys, denial codes, +typed/direct-SQL event parity, mutable-model service custody, warning and +exception retention, state-changing top-level and nested mappings, and mutable +JSON buffers. Every repair has focused behavior proof. + +Valid findings addressed: yes + +Open sub-agent sessions: none + +## Remaining Gate + +External GitHub checks, CodeRabbit review, and explicit human merge approval +remain pending. AUTH-05B and later authorization chunks remain inactive. diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-pr-trust-bundle.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-pr-trust-bundle.md new file mode 100644 index 00000000..7c3e6b02 --- /dev/null +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-pr-trust-bundle.md @@ -0,0 +1,133 @@ +# WS-AUTH-001-05A PR Trust Bundle + +## Chunk + +`WS-AUTH-001-05A` - Shared Audit Ownership And Append-Only Authority Evidence + +## Goal + +Add one typed, privacy-bounded, append-only authority evidence envelope to the +existing shared `audit_events` path without changing legacy task/checker +behavior or activating authorization decisions. + +## Human-Approved Intent + +The human approved AUTH-05A as one semantic chunk bounded by its allowed files, +non-goals, acceptance criteria, behavior evidence, and required reviews rather +than an arbitrary production-line ceiling. AUTH-05B remains separate. + +## What Changed + +- Migration `0018` extends the shared audit table with versioned authority + fields, closed registries, cross-field constraints, database-owned time, + authority-cause integrity, and append-only normal-DML triggers. +- A typed authority input and shared audit service enforce the same event, + reason, fact, reference, permission, denial, role, and project matrices. +- Task repository compatibility methods delegate to `AuditRepository` while + preserving caller session and transaction ownership. +- Behavior tests cover all 20 authority events through typed and direct-SQL + positive and negative paths, legacy compatibility, rollback, mutation + custody, non-retention, and migration downgrade guards. + +## Why It Changed + +Later authorization chunks require trustworthy evidence custody before actor, +grant, permission evaluation, route cutover, idempotency, or invalidation +consumers can be implemented. + +## Design Chosen + +The existing audit table remains the sole store. Application writes use one +typed service and shared repository. PostgreSQL independently enforces the +closed envelope so direct SQL cannot bypass privacy or integrity rules. +Rejected mappings and JSON are inspected from one stable snapshot, and service +re-admission never serializes untrusted mutated values before validation. + +## Alternatives Rejected + +- A parallel authority event table was rejected because it would split audit + custody and duplicate writer behavior. +- Application-only validation was rejected because direct SQL must enforce the + same boundary. +- Splitting typed and SQL matrices into mergeable partial states was rejected + because either half alone is an incomplete security boundary. +- AUTH-05B idempotency/replay work remains deferred to its own chunk. + +## Scope Control + +No route, dependency, middleware, permission evaluator, actor/grant lifecycle, +authorization cutover, idempotency record/replay path, invalidation consumer, +token verifier, workflow, dependency, or CI threshold was added or changed. + +## Product Behavior + +Legacy audit behavior and task/checker projections remain unchanged. New +authority evidence is internal foundation behavior only; it does not grant, +revoke, authorize, route, or invalidate product actions. + +## Acceptance Criteria Proof + +- All 20 registered events share typed and PostgreSQL positive/negative cases. +- System and project grant scopes, project/resource equality, UUID references, + registered tokens, denial codes, cause domain, self-cause, and append-only + custody have direct behavior proof. +- Malformed, scalar, duplicate-key, hostile mapping, mutable-buffer, secret + representation, and post-validation mutation cases retain no rejected value + and emit no serializer warning. +- Legacy rows survive upgrade/downgrade proof and existing repository behavior. + +## Tests And Checks Run + +- Exact migration proof: 1 passed. +- Audit suite: 10 passed. +- Audit plus delegation coverage: 11 passed at 94.55 percent. +- Two affected task lifecycle regressions passed. +- Ruff, 95.1 percent docstring coverage, test-delta, migration line length, + stale wording/auth/artifact, Markdown links, and diff integrity passed. +- Full local repository tests were intentionally not repeated; GitHub Backend + CI owns the full suite and repository-wide 78 percent coverage floor. + +## Test Delta + +Tests were added or strengthened only. No assertion, raises guard, skip, xfail, +coverage threshold, or exclusion was removed or weakened. + +## CI Integrity + +No workflow, package, dependency, coverage configuration, or test selection +file changed. GitHub checks must pass before merge. + +## Reviewer Results + +Exact SHA `44901286b5c867a414cc39a9ccff5307bd23ad52` passed senior engineering, +architecture, reuse/dedup, docs, product/ops, QA/test, CI integrity, test delta, +and security/auth review with no remaining findings. + +## External Review + +GitHub Actions and CodeRabbit are pending PR publication. + +## Remaining Risks + +- PostgreSQL append-only triggers protect normal DML, not database-owner or DDL + credentials; production must use the documented non-owner runtime role. +- The full repository suite and global coverage floor remain GitHub CI gates. +- AUTH-05A records evidence but intentionally does not consume invalidations or + perform authorization. + +## Follow-Up Work + +After merge, post-merge memory, and a separate human start, AUTH-05B may add the +idempotency and invalidation foundation. Later chunks own actor profiles, +grants, permission evaluation, and route cutover. + +## Human Review Focus + +Review the migration constraints and downgrade ordering, typed/SQL matrix +parity, single-snapshot privacy boundary, shared writer transaction ownership, +and proof that no authorization behavior was activated. + +## Human Merge Ownership + +Only the human may approve and merge this PR. Completion of internal or +external review does not authorize merge. From 957dd3d74566cbaf1cdca8470f83af0ed95bbbe0 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 14:11:33 +0100 Subject: [PATCH 20/22] fix(auth): address PR review feedback --- .agent-loop/REVIEW_LOG.md | 10 ++++ ...S-AUTH-001-05B-idempotency-invalidation.md | 24 ++++++++-- ...S-AUTH-001-05A-external-review-response.md | 47 +++++++++++++++++++ ...S-AUTH-001-05A-internal-review-evidence.md | 2 +- .../WS-AUTH-001-05A-pr-trust-bundle.md | 8 +++- backend/app/modules/audit/schemas.py | 9 +++- backend/tests/test_tasks.py | 3 +- 7 files changed, 94 insertions(+), 9 deletions(-) create mode 100644 .agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-external-review-response.md diff --git a/.agent-loop/REVIEW_LOG.md b/.agent-loop/REVIEW_LOG.md index 805ad098..6265331c 100644 --- a/.agent-loop/REVIEW_LOG.md +++ b/.agent-loop/REVIEW_LOG.md @@ -1,5 +1,15 @@ # Review Log +## 2026-07-14 - WS-AUTH-001-05A CodeRabbit Response + +CodeRabbit run `30676bdc-7525-4deb-8f9e-a87d42c64f92` produced four actionable +comments on PR #115. The response makes the inactive AUTH-05B verification +commands executable, replaces fragile positional event mapping with explicit +identity-link event keys, types the task audit fixture session, and replaces +misleading reused-agent names with AUTH-05A-specific review references. Focused +behavior, Ruff, stale authorization docs, Markdown links, and diff integrity +pass. Exact-SHA internal re-review is required before publication of the repair. + ## 2026-07-14 - WS-AUTH-001-05A Semantic Scope Amendment The human replaced AUTH-05A's numeric production-line ceiling with the approved diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05B-idempotency-invalidation.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05B-idempotency-invalidation.md index e77fddbe..a1fcf5d6 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05B-idempotency-invalidation.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05B-idempotency-invalidation.md @@ -147,11 +147,25 @@ result to an API error. Denials are not stored as successful replay results. ## Verification ```bash -(cd backend && .venv/bin/python -m ruff check app tests) -(cd backend && isolated PostgreSQL: pytest -q tests/test_alembic.py::<0019-proof> tests/test_authorization.py tests/test_audit.py) -(cd backend && focused AUTH-05B coverage >= 90 percent) -(cd backend && isolated full suite --cov=app --cov-fail-under=78) -(cd backend && .venv/bin/docstr-coverage --config .docstr.yaml) +cd backend +tmp_dir=$(mktemp -d) +trap 'rm -rf "$tmp_dir"' EXIT +.venv/bin/python -m ruff check app tests +.venv/bin/python scripts/run_isolated_tests.py \ + --metadata-json "$tmp_dir/05b.json" --timeout-seconds 1800 -- \ + .venv/bin/python -m pytest -q \ + tests/test_alembic.py::test_authority_idempotency_schema_preserves_audit_and_guards_downgrade \ + tests/test_authorization.py tests/test_audit.py +.venv/bin/python scripts/run_isolated_tests.py \ + --metadata-json "$tmp_dir/05b-coverage.json" --timeout-seconds 1800 -- \ + .venv/bin/python -m pytest -q tests/test_authorization.py tests/test_audit.py \ + --cov=app.modules.authorization --cov-report=term-missing --cov-fail-under=90 +.venv/bin/python scripts/run_isolated_tests.py \ + --metadata-json "$tmp_dir/05b-full.json" --timeout-seconds 10800 -- \ + .venv/bin/python -m pytest -q tests --cov=app --cov-report=term-missing \ + --cov-fail-under=78 +.venv/bin/docstr-coverage --config .docstr.yaml +cd .. python3 scripts/check_stale_workstream_wording.py python3 scripts/check_stale_authorization_docs.py python3 scripts/check_stale_artifact_contracts.py diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-external-review-response.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-external-review-response.md new file mode 100644 index 00000000..378a3a54 --- /dev/null +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-external-review-response.md @@ -0,0 +1,47 @@ +# WS-AUTH-001-05A External Review Response + +External review: CodeRabbit run `30676bdc-7525-4deb-8f9e-a87d42c64f92` +Pull request: `#115` + +## Comments Addressed + +1. Replaced `_REASONS` positional enum slicing with explicit identity-link + event keys while preserving `identity_lifecycle_change` behavior. +2. Added the `AsyncSession` annotation to the task audit owner-cleanup helper. +3. Replaced inactive AUTH-05B verification prose and placeholders with concrete + isolated migration, focused behavior, 90 percent subsystem coverage, and 78 + percent full-suite commands in the required order. +4. Replaced misleading reused AUTH-04B reviewer-session names with concrete + AUTH-05A final-review references tied to the reviewed code SHA. + +## Comments Deferred + +None. + +## Non-Actionable Review Output + +CodeRabbit's generic docstring warning reports a diff-local percentage rather +than the repository's configured gate. No docstring-only churn was added: the +actual `docstr-coverage --config .docstr.yaml` result passes at 95.1 percent. + +## Human Decisions Needed + +None. + +## Commands Rerun + +```text +pytest test_authority_input_rejects_unbounded_or_inconsistent_evidence +pytest test_task_repository_delegates_audit_persistence +ruff check backend/app/modules/audit/schemas.py backend/tests/test_tasks.py +python3 scripts/check_markdown_links.py +python3 scripts/check_stale_authorization_docs.py +git diff --check +``` + +Focused result: 2 passed. All listed static checks passed. + +## Remaining Risks + +GitHub Backend owns the full-suite and repository-wide coverage gate. AUTH-05B +remains an inactive contract; none of its runtime behavior is implemented. diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-internal-review-evidence.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-internal-review-evidence.md index 89bb2107..04092bb8 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-internal-review-evidence.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-internal-review-evidence.md @@ -2,7 +2,7 @@ Reviewed code SHA: `44901286b5c867a414cc39a9ccff5307bd23ad52` Reviewed at: `2026-07-14T12:45:18Z` -Reviewer run IDs: `/root/auth04b_final_eng`, `/root/auth04b_final_qa`, `/root/auth04b_final_security` +Reviewer run IDs: `AUTH-05A-4490128-eng-final`, `AUTH-05A-4490128-qa-final`, `AUTH-05A-4490128-security-final` ## Deterministic Evidence diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-pr-trust-bundle.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-pr-trust-bundle.md index 7c3e6b02..2082a5aa 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-pr-trust-bundle.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-pr-trust-bundle.md @@ -105,7 +105,13 @@ and security/auth review with no remaining findings. ## External Review -GitHub Actions and CodeRabbit are pending PR publication. +CodeRabbit run `30676bdc-7525-4deb-8f9e-a87d42c64f92` produced four +actionable comments. The response replaces positional reason mapping with +explicit event keys, types the task audit cleanup session, makes the inactive +AUTH-05B verification block executable, and records AUTH-05A-specific reviewer +references. The generic CodeRabbit docstring warning is not applicable because +the repository's configured docstring gate passes at 95.1 percent. GitHub +Backend CI remains the final full-suite gate. ## Remaining Risks diff --git a/backend/app/modules/audit/schemas.py b/backend/app/modules/audit/schemas.py index 62b77f0d..3433016f 100644 --- a/backend/app/modules/audit/schemas.py +++ b/backend/app/modules/audit/schemas.py @@ -102,7 +102,14 @@ class ActorReferenceKind(StrEnum): _REASONS = { AuthorityEventType.ACTOR_PROFILE_PROVISIONED: {"automatic_first_access"}, AuthorityEventType.SERVICE_ACTOR_PROVISIONED: {"manual_service_provisioning"}, - **dict.fromkeys(tuple(AuthorityEventType)[2:5], {"identity_lifecycle_change"}), + **dict.fromkeys( + ( + AuthorityEventType.ACTOR_IDENTITY_LINKED, + AuthorityEventType.ACTOR_IDENTITY_LINK_REVOKED, + AuthorityEventType.ACTOR_IDENTITY_LINK_REACTIVATED, + ), + {"identity_lifecycle_change"}, + ), AuthorityEventType.ACTOR_PROFILE_SUSPENDED: {"security_response", "administrative_correction"}, AuthorityEventType.ACTOR_PROFILE_REACTIVATED: {"administrative_correction"}, AuthorityEventType.ACTOR_PROFILE_DEACTIVATED: {"security_response", "administrative_correction"}, diff --git a/backend/tests/test_tasks.py b/backend/tests/test_tasks.py index ee1a1b81..89f70cd5 100644 --- a/backend/tests/test_tasks.py +++ b/backend/tests/test_tasks.py @@ -17,6 +17,7 @@ from sqlalchemy import inspect, select, text from sqlalchemy.dialects import postgresql from sqlalchemy.exc import IntegrityError +from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.schema import CreateIndex from app.adapters.auth.dev import actor_id_from_external_identity @@ -77,7 +78,7 @@ async def test_task_repository_delegates_audit_persistence() -> None: ) -async def delete_audit_fixture_as_owner(session, event_id: str) -> None: +async def delete_audit_fixture_as_owner(session: AsyncSession, event_id: str) -> None: """Construct missing-evidence corruption under explicit test-owner custody.""" await session.execute(text("lock table audit_events in access exclusive mode")) await session.execute( From ea16fd8bd2d9bc38b37c12003e51416c08a56678 Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 14:16:41 +0100 Subject: [PATCH 21/22] docs(auth): align 05B verification with CI --- .../chunks/WS-AUTH-001-05B-idempotency-invalidation.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05B-idempotency-invalidation.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05B-idempotency-invalidation.md index a1fcf5d6..7e399afc 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05B-idempotency-invalidation.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/chunks/WS-AUTH-001-05B-idempotency-invalidation.md @@ -160,9 +160,12 @@ trap 'rm -rf "$tmp_dir"' EXIT --metadata-json "$tmp_dir/05b-coverage.json" --timeout-seconds 1800 -- \ .venv/bin/python -m pytest -q tests/test_authorization.py tests/test_audit.py \ --cov=app.modules.authorization --cov-report=term-missing --cov-fail-under=90 +.venv/bin/python -m pytest -q tests/test_isolated_database_runner.py .venv/bin/python scripts/run_isolated_tests.py \ --metadata-json "$tmp_dir/05b-full.json" --timeout-seconds 10800 -- \ - .venv/bin/python -m pytest -q tests --cov=app --cov-report=term-missing \ + .venv/bin/python -m pytest -q tests \ + --ignore=tests/test_isolated_database_runner.py \ + --cov=app --cov-report=term-missing \ --cov-fail-under=78 .venv/bin/docstr-coverage --config .docstr.yaml cd .. From d0239529a1f9617d7db4d187dff2d273e97a4e3c Mon Sep 17 00:00:00 2001 From: Abiorh001 Date: Tue, 14 Jul 2026 14:21:10 +0100 Subject: [PATCH 22/22] docs(auth): record CodeRabbit repair evidence --- .../WS-AUTH-001-05A-external-review-response.md | 13 ++++++++++++- .../WS-AUTH-001-05A-internal-review-evidence.md | 16 ++++++++++------ .../reviews/WS-AUTH-001-05A-pr-trust-bundle.md | 10 ++++++---- 3 files changed, 28 insertions(+), 11 deletions(-) diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-external-review-response.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-external-review-response.md index 378a3a54..77700776 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-external-review-response.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-external-review-response.md @@ -14,6 +14,11 @@ Pull request: `#115` 4. Replaced misleading reused AUTH-04B reviewer-session names with concrete AUTH-05A final-review references tied to the reviewed code SHA. +Required internal QA then found that the new full-suite example recursively +included the isolation-runner self-tests. The command now runs those self-tests +directly and excludes only that file from the isolated full suite, matching +Backend CI without changing either coverage threshold. + ## Comments Deferred None. @@ -33,13 +38,19 @@ None. ```text pytest test_authority_input_rejects_unbounded_or_inconsistent_evidence pytest test_task_repository_delegates_audit_persistence +pytest test_authority_event_matrix_has_typed_direct_sql_parity +pytest tests/test_isolated_database_runner.py ruff check backend/app/modules/audit/schemas.py backend/tests/test_tasks.py +bash -n python3 scripts/check_markdown_links.py python3 scripts/check_stale_authorization_docs.py git diff --check ``` -Focused result: 2 passed. All listed static checks passed. +Focused result: 3 authority/delegation cases and the isolated-runner self-tests +passed. All listed static checks passed. Exact SHA +`ea16fd8bd2d9bc38b37c12003e51416c08a56678` then passed every required internal +review track with no remaining findings. ## Remaining Risks diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-internal-review-evidence.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-internal-review-evidence.md index 04092bb8..03e82b7f 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-internal-review-evidence.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-internal-review-evidence.md @@ -1,8 +1,8 @@ # WS-AUTH-001-05A Internal Review Evidence -Reviewed code SHA: `44901286b5c867a414cc39a9ccff5307bd23ad52` -Reviewed at: `2026-07-14T12:45:18Z` -Reviewer run IDs: `AUTH-05A-4490128-eng-final`, `AUTH-05A-4490128-qa-final`, `AUTH-05A-4490128-security-final` +Reviewed code SHA: `ea16fd8bd2d9bc38b37c12003e51416c08a56678` +Reviewed at: `2026-07-14T13:18:49Z` +Reviewer run IDs: `AUTH-05A-ea16fd8-eng-external-fix`, `AUTH-05A-ea16fd8-qa-external-fix`, `AUTH-05A-ea16fd8-security-external-fix` ## Deterministic Evidence @@ -46,7 +46,10 @@ Review cycles repaired valid findings covering system-scope SQL `NULL` semantics, canonical UUID event IDs, duplicate JSON keys, denial codes, typed/direct-SQL event parity, mutable-model service custody, warning and exception retention, state-changing top-level and nested mappings, and mutable -JSON buffers. Every repair has focused behavior proof. +JSON buffers. External review repairs removed positional event coupling, typed +the task cleanup session, and made the inactive AUTH-05B verification block +executable and consistent with Backend CI. Every repair has focused behavior +proof. Valid findings addressed: yes @@ -54,5 +57,6 @@ Open sub-agent sessions: none ## Remaining Gate -External GitHub checks, CodeRabbit review, and explicit human merge approval -remain pending. AUTH-05B and later authorization chunks remain inactive. +GitHub Backend and CodeRabbit follow-up checks plus explicit human merge +approval remain pending. AUTH-05B and later authorization chunks remain +inactive. diff --git a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-pr-trust-bundle.md b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-pr-trust-bundle.md index 2082a5aa..8124c4ba 100644 --- a/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-pr-trust-bundle.md +++ b/.agent-loop/initiatives/WS-AUTH-001-workstream-authorization-service/reviews/WS-AUTH-001-05A-pr-trust-bundle.md @@ -99,7 +99,7 @@ file changed. GitHub checks must pass before merge. ## Reviewer Results -Exact SHA `44901286b5c867a414cc39a9ccff5307bd23ad52` passed senior engineering, +Exact SHA `ea16fd8bd2d9bc38b37c12003e51416c08a56678` passed senior engineering, architecture, reuse/dedup, docs, product/ops, QA/test, CI integrity, test delta, and security/auth review with no remaining findings. @@ -109,9 +109,11 @@ CodeRabbit run `30676bdc-7525-4deb-8f9e-a87d42c64f92` produced four actionable comments. The response replaces positional reason mapping with explicit event keys, types the task audit cleanup session, makes the inactive AUTH-05B verification block executable, and records AUTH-05A-specific reviewer -references. The generic CodeRabbit docstring warning is not applicable because -the repository's configured docstring gate passes at 95.1 percent. GitHub -Backend CI remains the final full-suite gate. +references. Internal QA additionally aligned the full-suite example with +Backend CI by running isolation-runner self-tests outside the isolated suite. +The generic CodeRabbit docstring warning is not applicable because the +repository's configured docstring gate passes at 95.1 percent. GitHub Backend +CI remains the final full-suite gate. ## Remaining Risks