[anth-volk]

Fixes #4008

Summary

• The v2 table user_policies is renamed to user_policy_associations, matching user_household_associations, user_simulation_associations and user_report_associations. Its primary key, unique constraint, three check constraints, two foreign keys and two convention-named indexes take the new name; the foreign keys from legacy_user_policy_mappings and user_report_associations follow the table.
• Deliberate choice: the three composite listing indexes keep their names ix_user_policies_country_user_created_id, ix_user_policies_country_policy and ix_user_policies_country_legacy_policy. The sibling association tables keep short names for the same indexes (ix_user_households_*, ix_user_simulations_*, ix_user_reports_*), so the short names are the existing convention.
• Saved policies, their v1 mappings, the reports that name them and table grants are kept: the table is renamed in place, not recreated.
• A temporary view named user_policies with every column of the renamed table lets the Public API release from before the rename keep reading and writing saved policies, during the deploy and after a rollback to it. It is the approved hand-written exception described below. Removing it is a separate, later change, made once no deployable release names user_policies.
• The model, the table privilege declarations (privileges.py) and V2_ALEMBIC_HEAD_REVISION use the new name. The privilege manifest declares the view with its table's privileges (TEMPORARY_VIEW_TABLES), and the privilege reconciliation now finds, grants and verifies privileges on views (its existing-relation query read only ordinary and partitioned tables before). It reads tables and views separately: a declared table must exist as a table and a declared temporary view as a view, otherwise it is reported as missing ("Declared tables missing" or "Declared views missing") and nothing is granted. The Stage 10 saved-policy dual-write copy goes through the model and follows it.
• Migration qualification commands. Both run before every schema upgrade, so they read the saved-policy table under whichever name the database has. The policy qualification counts only while its revision is pending, when the table is still user_policies, so it names that table. The user-association qualification counts on every run, so it counts user_policy_associations when that table exists and user_policies otherwise (views are not listed as tables, so the view is never counted). Their JSON evidence keeps the key user_policies.
• Unchanged: the route /v2/user-policies, the user_policy_id columns, legacy_user_policy_mappings, and the v1 Cloud SQL table user_policies.

Migration notes

• Route groups covered: none change. The policy user-association routes keep their contracts; only the table behind them is renamed. The generated migration contracts do not change.
• What stays on the Flask/v1 path: everything; v1's Cloud SQL user_policies table is untouched.
• Newly prepared: one Alembic v2 revision, fa7fa1035444 (parent fbb7488ebe8a).
• User-visible API contract changes: none.

The migration correction, and why it is within the migration rules

The revision was generated with uv run alembic -c alembic-v2.ini revision --autogenerate. Autogeneration cannot detect a rename: it created an empty user_policy_associations table and dropped user_policies, which would delete every saved policy, and its copies of the three composite indexes would have collided with the old table's indexes of the same names. The repository's Alembic rules allow a minimal, documented, tested post-generation correction of an objectively incorrect autogenerated operation. The revision has one such correction in each direction:

• op.rename_table replaces the create-and-drop pair.
• Each constraint the generated table named after user_policy_associations is dropped under its old name and created under the generated name with the generated definition, using Alembic operations only.
• The generated drops and re-creations of the two convention-named indexes and of the three foreign keys that refer to the table are kept, ordered around the rename. The downgrade is the exact reverse.

Rebuilding the primary key, unique and check constraints takes a brief lock on the table during the upgrade.

The temporary user_policies view: an approved hand-written exception

Creating a view is a hand-written schema operation, which the migration rules leave to a human decision. A maintainer approved this one on 2026-10-22. It is the only hand-written SQL in the v2 chain:

• After the rename, the upgrade runs CREATE VIEW user_policies AS SELECT ... FROM user_policy_associations, listing every column in the table's order. The downgrade runs DROP VIEW user_policies before renaming the table back. Both are marked in the revision as the approved exception, and docs/engineering/skills/alembic-migrations.md ("API v2 saved-policy table rename") documents them.
• test_alembic_v2.py allows exactly these two statements and fails if any other v2 revision uses op.execute.
• A single-table view is automatically updatable, so SELECT, INSERT ... RETURNING, UPDATE, DELETE and row locks through it act on the table, with the table's defaults, constraints and cascades. The release before the rename uses no ON CONFLICT on saved policies.
• Grants: the runtime role gets SELECT, INSERT, UPDATE, DELETE on the view, as on the table, through the privilege reconciliation that runs after every upgrade. The view runs with its owner's (the migration role's) privileges on the table.
• Autogeneration and schema checks: Alembic does not reflect views, so alembic check reports no operations and the head schema validation (which compares tables) passes with the view present; the lifecycle test checks both.
• Until the view is removed, a later revision cannot drop or change the type of a user_policy_associations column the view shows without dropping the view first.

What the release before the rename can do:

• During the deploy (from the schema upgrade in the database job until the new release takes traffic) and after an application rollback to it: everything it does with saved policies today, through the view. That covers its /v2/user-policies routes (create, list, read, rename, repoint, delete), report creation that checks the saved policies a report names, the sign-in merge that moves an unauthenticated user's saved policies, account export, and the Stage 10 saved-policy dual-write copy (DB_WRITE_POLICY is cloud_sql today, so that copy is off).
• One difference: an error raised by a renamed constraint now carries the constraint's new name. No code maps saved-policy errors by constraint name, so responses are unchanged.
• After the view is removed by the later change, such a release fails every statement on saved policies. Downgrading this revision alone drops the view and renames the table back with every row, reference and table grant.

The Stage 12A document (docs/migration/stage-12a-user-login.md) says this in its rollback and schema compatibility sections, and uses the new table name for current behavior.

Open risk for a person to decide: the view's grants just after the upgrade

In the deploy's database job (seed-v2-database.yml), the step "Upgrade and verify the v2 schema" runs alembic upgrade head (which commits the rename and the view), then alembic current --check-heads and alembic check. Only the next step, "Grant declared v2 table privileges" (reconcile_v2_privileges.py apply), grants the runtime role its privileges on the view. In that interval, from the upgrade's commit until the grant step commits (normally the few seconds the three uv run commands take, longer if a step fails and is re-run), the release still serving uses saved policies through the view as policyengine_v2_runtime.

• If the project's default privileges give the runtime role SELECT, INSERT, UPDATE and DELETE on relations the migration role creates in public, the view has them from the moment it is created: PostgreSQL applies ON TABLES default privileges to views too (checked on a disposable database). docs/migration/stage-8-supabase-target.md says each project has such defaults ("tables the migration role creates give the runtime role read/write access"), but they are set outside the repository; the reconciliation only reports them in the job summary ("Default privileges for new tables"), and no test or deploy step checks them.
• If they do not, every saved-policy statement from the release still serving fails with "permission denied for view user_policies" during that interval: its /v2/user-policies routes, report creation that names a saved policy, the sign-in merge and the Stage 10 saved-policy copy.

Two options, for a person to choose:

1. Rely on the default privileges: an operator confirms, in staging and production, that the migration role's default privileges in public give policyengine_v2_runtime SELECT, INSERT, UPDATE and DELETE on tables (the last deploy's job summary lists them).
2. Add GRANT SELECT, INSERT, UPDATE, DELETE ON user_policies TO policyengine_v2_runtime to the revision, right after the view is created. That is a second hand-written statement beyond the approved view, so it needs the same approval.

This PR does neither.

Environment variables

The change adds no environment variable. It changes one workflow step: "Test signed-in identity provisioning and user associations" in .github/workflows/v2-integration-check.yml (job catalog-runtime-integration, called from pr.yml and push.yml) now also runs tests/integration/test_v2_saved_policy_view.py. That step receives the job-level env: block; every value is a literal in the workflow, pointing at the job's own service containers, and no secret reaches it:

Variable	Value and provider	Read by this step
V2_MIGRATION_DATABASE_URL	Literal, the job's disposable postgres:17 service	Yes (tests/integration/conftest.py and both modules)
V2_ALEMBIC_DISPOSABLE_TEST	Literal "1"	Yes
DEPLOYMENT_ENVIRONMENT	Literal test (names the Stage 12 role policyengine_stage12_test)	Yes (the managed-roles fixture)
ALEMBIC_DATABASE_URL	Literal, the job's disposable mysql:8.4 service	Available, not read by these modules
RUNTIME_CACHE_TEST_URL	Literal, the job's redis:7.2-alpine service	Available, not read by these modules

CODECOV_TOKEN (GitHub secret, passed with secrets: to the reusable workflow) is read only by the coverage upload step, which this PR does not change. The rule api.v2.runtime-privileges-tested-as-runtime-role now names the new module in its scope.

What this PR waits for

• Let saved reforms and households point at a new v2 policy or household #4005, its base (the bottom of this stack of Public API PRs).
• Review, including a core-developers review for the workflow and rules/ changes.
• A person's decision on the open risk above (default privileges verified by an operator, or an approved GRANT in the revision).
• Alembic v2 chain coordination: if another PR with a v2 revision merges first, this revision is rebased onto the new head (one revision merges at a time).

Testing

CI runs on this branch are cancelled at the user's request while the stack is prepared; the local runs below are the evidence. The branch is rebased onto #4005's branch, which is rebased onto master at b5cf71dc8 (2026-10-26). Each commit was written test-first; the SHAs are after the rebase:

• 3d0b10533 (qualification commands): test_saved_policies_are_counted_under_their_name_after_the_rename failed first (assert 0 == 3: the renamed table was never counted). Then both qualification unit files passed (16) and the qualification integration test passed against disposable PostgreSQL 17. The two tests for the pending case pin the old name; they passed before and after.

• 320c5d20d (the rename): on a fresh disposable database, 11 unit tests failed first (KeyError on user_policy_associations, the missing revision, the privilege manifest, the recorded saved-policy SQL) and 4 lifecycle tests failed (NoSuchTableError public.user_policy_associations, user_policies still present). Then they passed, with alembic check clean, privilege apply/verify, the v2 and v1 integration tests, and the full make test set.

• 54bef66a1 (review follow-up, tests): with a placeholder later revision added locally, the old revision test read the wrong file and failed, and the new one passed. has_table_privilege(role, table, 'SELECT, UPDATE') returned true for a role holding only SELECT, so each privilege is now checked on its own. With ON DELETE SET NULL removed from the downgrade's report foreign key (locally, not committed), the new downgrade cascade check failed.

• c747f002e (review follow-up, document only): the Stage 12A document.

• d31f13df2 (reconciliation and views): a new integration test that declares privileges on a view failed first (the view was reported as a missing table, so nothing was granted); then the reconciliation integration and unit tests passed (63).

• 035af26e0 (the view): before the view existed, the revision tests failed (no approved exception in the revision), the manifest test failed to import TEMPORARY_VIEW_TABLES, 3 lifecycle tests failed (relation "user_policies" does not exist, no view at head) and the 4 runtime-role view tests failed (relation "user_policies" does not exist). Then they passed: the lifecycle test checks the view's columns and absence after the downgrade and runs the parent revision's inserts, reads, updates and deletes through it; test_v2_saved_policy_view.py runs the saved-policy services, the sign-in merge, report creation and the Stage 10 copy as the runtime role with every statement naming user_policies, and checks the rows in user_policy_associations.

• eb4c29c1d (review follow-up: the view tests in CI): a new unit test, test_every_managed_role_integration_module_runs_in_ci_and_is_in_its_rule, failed first (test_v2_saved_policy_view.py was neither in the workflow nor in the rule's scope); then the workflow tests passed (20). The changed step's command, uv run coverage run -a --branch -m pytest -q tests/integration/test_v2_user_identity_persistence.py tests/integration/test_v2_saved_policy_view.py, run locally after the job's earlier steps (v1 and v2 upgrade head, privilege apply --create-missing-roles and verify) with the job's variables pointing at local disposable databases: 20 passed.

• c0794eb23 (review follow-up: relation kinds): new unit and integration tests failed first (plan_reconciliation() and apply_declared_privileges() had no way to name views, so a view or table of either kind satisfied any entry); then the reconciliation unit tests (63 with the manifest tests) and integration tests (5) passed. With the database at fbb7488ebe8a, apply now reports "Declared views missing from the database (nothing was granted): user_policies" and exits 1 instead of counting the old table as the view.

On the final branch, against disposable PostgreSQL 17 and MySQL 8.4:

• alembic -c alembic-v2.ini upgrade head and check (no new operations); reconcile_v2_privileges.py apply --create-missing-roles (granted the runtime role DELETE, INSERT, SELECT, UPDATE on user_policies) and verify.
• The v2 lifecycle, view, privilege reconciliation, identity, comparison-run, saved-policy mirroring, policy and household persistence integration tests: 52 passed.
• The MySQL v1 lifecycle (4) and the v1 policy, saved-policy and household dual-write integration tests (29) passed.
• The make test set without coverage: 3,411 passed.
• ruff check ., ruff format --check ., uv run --frozen --extra dev mypy, python scripts/run_quality_guards.py, python -m scripts.rules validate and guard passed; python scripts/export_migration_contracts.py left the generated files unchanged.

No inline # noqa suppressions are added. A rules fixture (rules/fixtures/api.v2.runtime-privileges-declared/reads_good.py) changes its docstring to the new table name, so this PR requests a review from core-developers.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 91.66667% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 87.95%. Comparing base (8b9272d) to head (459b031).

Files with missing lines	Patch %	Lines
...ata/v2/user_association_migration_qualification.py	85.71%	1 Missing ⚠️

Additional details and impacted files

@@                          Coverage Diff                          @@
##           v2plan/1f-121-repoint-saved-items    #4010      +/-   ##
=====================================================================
- Coverage                              87.95%   87.95%   -0.01%     
=====================================================================
  Files                                    259      259              
  Lines                                  16169    16174       +5     
  Branches                                2636     2636              
=====================================================================
+ Hits                                   14222    14226       +4     
- Misses                                  1275     1276       +1     
  Partials                                 672      672              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[vahid-ahmadi]

Review at c0794eb239.

Verdict: correct and safe for data; mergeable once the grant question in the description is settled. Stacked on #4005, so I reviewed only v2plan/1f-121-repoint-saved-items...v2plan/1f-122-rename-user-policies. No must items.

1. should On the open grant question: I would take option 2 (add the GRANT to the revision). Option 1 depends on default privileges that live outside the repo, no deploy step checks them, and v2/user-policies returns 503 for logged-in users: reforms cannot be saved #3977 shows they have been wrong before. With the grant in the same transaction as CREATE VIEW, the view is never unusable by the live release, whatever the defaults are.
2. should migrations/v2/versions/fa7fa1035444_rename_user_policies_to_user_policy_.py:159: the view is created without security_invoker, so it runs with the migration role's rights on the table. That makes no difference today, as nothing in the repo turns on row-level security. But this is a Supabase project, and Supabase's database linter flags security-definer views in public. Worth checking in staging that anon and authenticated hold no privilege on user_policies (reconciliation never revokes, and it checks only the roles it manages). Alternatively, create the view WITH (security_invoker = true) (PostgreSQL 15+), which changes nothing for the runtime role because it already has the same grants on the table.
3. nit The foreign key name fk_user_report_associations_baseline_user_policy_id_user_policy_associations is 76 characters, so PostgreSQL stores a truncated name. It round-trips, because the upgrade and the downgrade both go through op.f() and alembic check is clean, but anyone looking it up in pg_constraint by its full name will not find it.

Checked and fine: rename_table keeps every row, mapping and report reference; the downgrade drops the view and renames back, with no data loss; the reconciliation's relation-kind query uses bound parameters, with no string-built SQL; only the two approved op.execute statements are present.

What I ran: fresh clone, uv sync --frozen --extra dev, then pytest on the changed unit test files: 186 passed. CI was cancelled on this branch, and I did not run the integration tests (test_v2_saved_policy_view.py, lifecycle, reconciliation), which need disposable PostgreSQL. So the view and grant behaviour rest on the author's local runs until CI runs.

🤖 Generated with Claude Code