Skip to content

Remove the routes only the legacy site used, check health at /health, and stop serving Canada, Nigeria and Israel - #4017

Draft
anth-volk wants to merge 5 commits into
masterfrom
v2plan/1g-128-remove-unused-routes
Draft

anth-volk wants to merge 5 commits into
masterfrom
v2plan/1g-128-remove-unused-routes

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #4016

Do not merge yet. This PR may merge only after legacy.policyengine.org is no longer served, after Cloud Run request logs show no remaining traffic to the routes and countries it removes, and after the external uptime monitor that polls GET / is moved to /health. #4012, which it was stacked on, has merged, and it now targets master.

Summary

  • Routes only the legacy site or the documentation used are removed, from both the Flask application and the native routes: the v1 saved-policy routes (POST, GET, PUT /{country_id}/user-policy), the user-profile routes (POST, GET, PUT /{country_id}/user-profile), the policy search (GET /{country_id}/policies), the recent-simulations list (GET /simulations), the GET / landing page and GET /specification with its hand-written OpenAPI document. Code only they called goes too. POST /{country_id}/calculate stays: it served about 140,000 requests in 30 days, and its callers move first. The v1 saved-policy service and copying modules stay until dual-write is deleted.
  • Health checks use /health. /health answers 503 {"status": "not_ready"} until the readiness probe passes (startup warmup finished, runtime configuration valid) and 200 {"status": "healthy"} after, taking over from /readiness-check. The Cloud Run startup probe and the seven health checks in push.yml (candidates, stable URLs and revision restores) poll it; a previous revision also answers 200 on /health once up, so a restore still passes its check. /liveness-check and /readiness-check are removed. This repository configures no load balancer health check: the public load balancer's backend is a serverless network endpoint group, which does not use one.
  • Canada, Nigeria and Israel are no longer served. Their requests get the unknown-country 400 response before any read or write, and their model packages are no longer installed (uv.lock loses them and 13 packages only they needed). COUNTRIES, the package-name lists in constants.py and the loaded country models now derive from the one country list in policyengine_api/supported_countries.py.

Request logs (production, 30 days to 2026-10-19): user-policy and user-profile 0; GET /{country_id}/policies 13; GET /simulations 11; GET / about 24,300 (an uptime monitor and link previews); the three countries about 1,160 (784 from legacy.policyengine.org, 49 from app.policyengine.org, which PolicyEngine/policyengine-app-v2#1390 stops).

Since legacy.policyengine.org stopped serving (2026-10-30, 15:55 UTC), production request logs to 17:15 UTC show no requests to user-policy, user-profile, GET /{country_id}/policies, GET /simulations, /specification, /liveness-check or /readiness-check. GET / had 46 (the uptime monitor and link previews), and the three countries had 9 (8 from a local test run). These counts need more days before this PR merges.

Migration notes

  • Route groups covered: health, specification, policy (saved policies and search), user_profile (v1 profile routes), metadata (country list).
  • What stays on the Flask/v1 path: every other v1 route, including POST /{country_id}/calculate.
  • User-visible API changes: the removed routes and countries answer 404 or the unknown-country 400; /health answers 503 while an instance is not ready. ROUTE_IMPL_HEALTH and ROUTE_IMPL_SPECIFICATION no longer select a route but stay required by the deployment pipeline until the Flask fallback is removed.
  • The app-v2 contract registry drops the removed requests and the generated contract documents are refreshed.

Rules

The session-ownership rule's enforcer now names test_flask_routes_do_not_manage_sessions (the check covers every Flask route module, instead of only the removed recent-simulations module). This changes rules/registry.toml, so it needs a review from a core developer other than the author.

Inline suppressions

None added.

Environment variables

  • The startup probe (CLOUD_RUN_STARTUP_PROBE) keeps its existing provider: the default in .github/scripts/cloud_run_env.sh, which no workflow overrides; only the probed path changes. deploy_cloud_run_candidate.sh passes it to gcloud run deploy as before.
  • The health checks in push.yml change only the URL path they poll; their inputs (steps.candidate.outputs.url, the stable URLs and the revisions to restore) come from the same earlier steps as before.
  • /health now reads the same readiness inputs /readiness-check read (policyengine_api.readiness.validate_policy_runtime_configuration: the v1 source selectors, the v2 runtime database settings, user authentication and Auth0 management settings, and the SPM selection), all from the existing Cloud Run environment set by deploy_cloud_run_candidate.sh. Staging and production already pass this check on every deploy through /readiness-check. Not verified beyond that: staging has no Auth0 management settings while production does; the check accepted that configuration on /readiness-check, and /health uses the same function.
  • The cross-database integration workflow already runs uv sync --frozen --extra dev on master; this PR only makes its workflow test require the dev extra, which the workflow needs once policyengine-canada no longer brings coverage in. It reads no new variables (its database URLs are literals for its own service containers).
  • No new variables, secrets or Terraform. Removing the three packages changes the image contents only; the Cloud Run image installs with uv sync --frozen --no-dev, which never included the dev tools.

What this PR waits for

  1. legacy.policyengine.org no longer served (an operator step). Done on 2026-10-30.
  2. Cloud Run request logs showing no traffic to the removed routes and the three countries.
  3. The external uptime monitor moved from GET / to /health (an operator step), and any other external monitor or probe that polls /liveness-check or /readiness-check moved too.
  4. Offer only the US and the UK in the calculator policyengine-app-v2#1390 merged, so the calculator no longer requests the three countries. (Add a v2 countries table that matches one country list in code, GET /v2/countries and a startup check #4012 merged on 2026-10-30.)
  5. Review, including a core developer for the rule change.

Testing

Each commit was written test-first:

  • 054ad2f0b (route removal): 4 tests failed first (the removed routes were still registered in the Flask blueprints and assembled by the app module, and the ASGI app still served /specification); then 217 passed in the route boundary, ASGI factory, request lifecycle, contract and runtime-cache tests.
  • a9003ffad (health checks): 26 tests failed first across the ASGI factory, deploy scripts, push workflow, boot smoke test, runtime measurement, request lifecycle, route boundaries and baseline capture; then 770 passed in those suites and the routes, contract and to-refactor API tests.
  • 630ca6461 (countries): 17 tests failed first (lists, dependencies and current-law IDs still named the three countries, their writes reached the services, and a country added to the one list did not reach the v1 lists); then the unit, contract and to-refactor suites passed (3,304).
  • 72f8090be (review follow-up): 2 tests failed first (HEAD /health answered 404); then 633 passed in the affected suites.
  • b54882286 (CI follow-up): dropping policyengine-canada also drops coverage from the environment uv sync --frozen installs (it was the only default dependency that pulled it in), and the cross-database integration workflow runs its tests under coverage. master already installs the dev extra there; the workflow test now requires it.
  • Rebase onto master (2026-10-30). The counts above were measured before the rebase. Tests master added since then used code this PR removes, and failed on the rebased branch (12 cases); each was fixed in the commit that removes that code. The local-area breakdown tests read the impact cache directly instead of the removed get_recent_reform_impacts, the parameter-path tests query the policy table instead of the removed policy search, and the release-test shard weights drop the deleted saved-policy route tests. The /health changes to the deployment-script tests moved into the files master split them into, and uv.lock was relocked from master's (it still loses only the three packages and the 13 only they needed). All 12 cases pass, and ruff format --check ., ruff check ., mypy, python scripts/run_quality_guards.py, the rules guard and uv lock --check pass on the rebased head.
  • Against disposable MySQL 8.4 and PostgreSQL 17: tests/integration/test_v2_country_routes.py, test_v2_metadata_routes.py, test_v1_household_dual_write.py and test_v1_policy_dual_write.py pass (23).
  • On every commit: ruff format --check ., ruff check ., uv run --frozen --extra dev mypy, python scripts/run_quality_guards.py, python -m scripts.rules guard --base-ref origin/master; uv lock --check passes after the lock change.

One review round found no bugs; its follow-ups are the fourth commit. After the rebase, two review rounds on the conflict resolution: the first found the test changes described above, and the second found no bugs.

🤖 Generated with Claude Code

@vahid-ahmadi

Copy link
Copy Markdown
Contributor

Review at 379885d41f.

Verdict: the change itself looks correct; it is ready once the four operator steps in the description are done and #4012 has merged. Stacked on #4012, so I reviewed only v2plan/1g-126-countries-table...v2plan/1g-128-remove-unused-routes. No must items.

  1. should policyengine_api/fastapi_routes/health.py:62: /health now returns 503 whenever readiness_probe() fails, at any point in the instance's life, not only during warmup. That is right for the startup probe and the deploy checks, but once the uptime monitor moves to /health, an invalid runtime setting (for example a v2 secret that stops resolving, which Add a v2 countries table that matches one country list in code, GET /v2/countries and a startup check #4012's docstring says readiness reports) will page as "API down" while v1 is still serving. Worth deciding whether the external monitor should alert on that, or whether it needs a separate always-200 liveness path. Either way, a sentence in docs/migration/cloud-run-operations.md would help whoever moves the monitor.
  2. nit Outside this repo: PolicyEngine/policyengine-skills skills/policyengine-api/SKILL.md:44 (and the same file in policyengine-claude) still lists /specification, /liveness-check and /readiness-check. Worth a follow-up so agents stop pointing people at removed routes.

Checked and fine:

  • I searched the PolicyEngine org for callers of the removed routes. policyengine-app-v2 uses user-policy-associations (a different path), not /{country_id}/user-policy. I found no callers of user-profile, the policy search or /simulations against this API (policyengine-chat calls /simulations/ on v2.api.policyengine.org, a different service).
  • The restore steps poll /health on the previous revision, which already answers 200 there, so a restore still passes.
  • COUNTRIES, the package lists and the current_law_id now come from the one list, so v1 and v2 cannot disagree.

What I ran: fresh clone, uv sync --frozen --extra dev on this head, then pytest on every added or changed file under tests/unit and tests/contract: 681 passed. CI does not run here because the PR is stacked (the CI complete gate fails on purpose), so the full suite, the Docker build and the integration tests have not run on this head. I did not read the 1,344-line openapi_spec.yaml deletion or the deleted tests line by line.

🤖 Generated with Claude Code

Base automatically changed from v2plan/1g-126-countries-table to master October 30, 2026 17:03
@anth-volk
anth-volk force-pushed the v2plan/1g-128-remove-unused-routes branch from 379885d to a073104 Compare October 30, 2026 17:38
@codecov

codecov Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 69.23077% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 51.45%. Comparing base (424cf47) to head (b548822).
⚠️ Report is 12 commits behind head on master.

Files with missing lines Patch % Lines
policyengine_api/country.py 0.00% 2 Missing ⚠️
policyengine_api/fastapi_routes/health.py 71.42% 2 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (424cf47) and HEAD (b548822). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (424cf47) HEAD (b548822)
2 1
Additional details and impacted files
@@             Coverage Diff             @@
##           master    #4017       +/-   ##
===========================================
- Coverage   88.11%   51.45%   -36.67%     
===========================================
  Files         285      267       -18     
  Lines       16577    15026     -1551     
  Branches     2712     2399      -313     
===========================================
- Hits        14607     7731     -6876     
- Misses       1292     6656     +5364     
+ Partials      678      639       -39     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

anth-volk and others added 5 commits October 30, 2026 21:52
Remove the v1 saved-policy routes (POST, GET and PUT
/{country_id}/user-policy), the user-profile routes (POST, GET and PUT
/{country_id}/user-profile), the policy search (GET /{country_id}/policies),
the recent-simulations list (GET /simulations), the GET / landing page and
GET /specification, in both the Flask application and the native FastAPI
routes. Cloud Run request logs for the 30 days to 2026-10-19 show no
saved-policy or user-profile requests and a few dozen manual or
legacy.policyengine.org requests to the others; GET / is polled by an
external uptime monitor that must move to /health before this merges.
POST /{country_id}/calculate stays: it served about 140,000 requests in the
same period.

Code only these routes called goes too: the home, user-profile and
recent-simulations blueprints, the v1 user-profile service, the policy
search and recent reform impacts service methods, the legacy OpenAPI
document and its loader, and the native route dependency that served it.
`ROUTE_IMPL_SPECIFICATION` is still read because the deployment pipeline
requires and verifies it, but it selects no route. The v1 saved-policy
service and copying modules stay until dual-write is deleted.

The app-v2 contract registry drops the removed requests, the generated
migration contract artifacts are regenerated, the Cloud Run candidate check
asserts the removed paths return 404, the session-ownership rule's route
check covers every Flask route module, and the SPM guide no longer points
at the removed specification document.

Tests (failed first: the removed routes were still registered in the Flask
blueprints and assembled by the app module, and the ASGI app still served
/specification): the route boundary tests, the ASGI factory, request
lifecycle, contract and runtime-cache tests (217 passed); the unit,
contract and to-refactor suites (3,297 passed once the log tests below
were fixed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…iness checks

`/health` now answers 503 `{"status": "not_ready"}` until the readiness
probe passes, that is until the startup warmup has finished and while the
runtime configuration is invalid, and 200 `{"status": "healthy"}` after. It
takes over the behaviour of `/readiness-check`, which held traffic off a
booting instance and stopped a deployment whose configuration could not
serve requests.

The Cloud Run startup probe that deploy_cloud_run_candidate.sh sets (the
default in cloud_run_env.sh) and the seven health checks in push.yml (staging
and production candidates, stable URLs, and the revision restores) now poll
`/health`. Every earlier revision also serves `/health` and returns 200 once
up, so restoring a previous revision still passes its health check. This
repository configures no load balancer health check: the public load
balancer's backend is a serverless network endpoint group, which does not
use one.

`/liveness-check` and `/readiness-check` are removed from the Flask
application and the native routes. `ROUTE_IMPL_HEALTH` no longer selects a
route but stays required by the deployment pipeline until the Flask fallback
is removed. The migration baseline probes, the runtime measurement script
and the local boot smoke test (which now starts the ASGI app and checks that
`/health` reports ready) use `/health`, the Cloud Run candidate check
asserts the removed paths return 404, a test requires every push workflow
health check to poll `/health`, and the operations, SPM and runbook
documents name the new path.

Tests (failed first: 26 across the ASGI factory, deploy scripts, push
workflow, boot smoke test, runtime measurement, request lifecycle, route
boundaries and baseline capture, because `/health` ignored readiness, the
probes and workflow still used the removed paths, and the routes were
still registered): then 770 passed in those suites and the routes,
contract and to-refactor API tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The v1 routes' country list (`COUNTRIES` in policyengine_api/constants.py),
the package-name lists beside it and the loaded country models
(policyengine_api/country.py) now derive from the one country list in
policyengine_api/supported_countries.py, so they hold the US and the UK.
Canada, Nigeria and Israel lose their economy options and current-law
policy IDs (policyengine_api/country.py and
policyengine_api/utils/get_current_law.py), and `policyengine_canada`,
`policyengine-ng` and `policyengine-il` leave pyproject.toml. uv.lock is
regenerated: it loses the three packages and the thirteen packages only they
needed, and nothing else changes.

Requests for the three countries now get the shared unknown-country 400
response ("Available countries are: uk, us") before any read or write, and
the API process no longer loads their tax-benefit systems. Cloud Run request
logs for the 30 days to 2026-10-19 show about 1,160 requests for them, 784
with a legacy.policyengine.org referer and 49 from app.policyengine.org,
which the app change removing them stops. Existing Cloud SQL rows for them
are kept.

Tests (failed first: 17, because the lists, dependencies and current-law
IDs still named the three countries, their requests reached the services,
and a country added to the one list did not reach the v1 lists): the served
list, dependencies and lockfile, the unknown-country response, the
rejection of their policy and household writes before any write, the
loaded models matching the served list, and, in a fresh interpreter, a
country added to the one list reaching every v1 list. The unit, contract
and to-refactor suites pass (3,304).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review follow-ups (no bugs found):

- `/health` was a GET route, so a HEAD request fell through to the Flask
  application and got a 404, while the removed Flask liveness check
  answered HEAD with 200. Uptime monitors and `curl -I` (used by two
  runbooks) send HEAD, so `/health` now answers it too, with the same
  status as GET and no body.
- The metadata document takes its current-law policy ID from
  `get_current_law_policy_id` instead of a second copy of the table.
- The country list's docstring says that API v1 loads each listed
  country's package and needs its current-law ID.
- Comments say why the route-group tables still name removed paths: so
  requests that still arrive for them are labelled in the request logs.

Tests (failed first: HEAD /health answered 404 whether or not the app was
ready): HEAD /health answers 200 when ready and 503 before, with no body;
the ASGI factory, request lifecycle, country, metadata, migration-flag,
route, contract and served-country tests pass (633).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Dropping the Canada package also drops `coverage` from the environment
`uv sync --frozen` installs: `policyengine-canada` was the only default
dependency that pulled it in. The cross-database integration workflow runs
every test step under `uv run coverage`, so it needs the dev extra, which
declares `coverage` and `pytest`. The workflow already installs it with
`uv sync --frozen --extra dev`; the workflow test now requires that
command instead of accepting any `uv sync --frozen`, so removing the extra
fails the test rather than the workflow.

Tests: the workflow test passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk
anth-volk force-pushed the v2plan/1g-128-remove-unused-routes branch from a073104 to b548822 Compare October 30, 2026 17:53

This branch was successfully deployed

1 active deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove the routes only the legacy site used, check health at /health, and stop serving Canada, Nigeria and Israel

2 participants