[anth-volk]

Fixes #984

Merge order: this is step 1 of 3 of removing x and y from UK constituency and local-authority records. #919 (step 2, the executor stops sending them) merges only after this is deployed, and then has main merged in, so its diff keeps only the executor change. #924 (step 3, the contract removes them) follows #919.

Same-Repository Draft PR Check

• This PR is a draft.
• This PR is from a branch in PolicyEngine/policyengine-sim-api, not a personal fork.

Changelog

• This PR adds a changelog fragment, changelog.d/<name>.<type>.md, whose type is breaking, added, changed, fixed or removed: changelog.d/984.changed.md.

Summary

The contract and the gateway accept UK constituency and local-authority records with or without x and y, each area's integer map position. The executor doesn't change and still sends both.

Why this is its own deploy

• On main, x and y are int | None with no default, so every record must carry both keys, and MacroOutputModel forbids unknown keys.
• The gateway validates each executor result against the contract when a job is polled (JobStatusResponse in job_status_routes.py). The entrypoint relays the gateway's response without validating it.
• A push to main deploys the executor before the gateway (deploy_gateway needs deploy_executor in simulation-deploy.reusable.yml). In the prod deploy of 30 Oct, the executor was live at 18:08:58 UTC and the gateway at 18:11:26 UTC.

If one deploy both made the fields optional and stopped the executor sending them, the old gateway would report every UK local-area breakdown polled in that gap as failed. After this PR is deployed, the gateway accepts records either way, so #919's deploy has no such gap.

What changes

• x and y on ConstituencyImpactRecord and LocalAuthorityImpactRecord default to None and use pydantic's field-level exclude_if. A record without them validates, and every dump leaves the keys out instead of writing null. Records with integer x and y still validate and keep them.
• Leaving the keys out, rather than writing null, matters for step 3: once Remove x and y from the contract's UK area records #924 removes the fields, the gateway refuses "x": null as an unknown key, so results Stop sending x and y on UK area results #919's executor returns must not carry it.
• The contract's pydantic floor rises to 2.12, the first release with exclude_if. Only that specifier changes in the five locks that record the contract's metadata.
• The gateway's golden OpenAPI no longer lists x and y as required, and the constituency description is updated.

This is #919's first commit (5b5fbde0), rebuilt on main:

• its gateway tests are in test_simulation_job_status.py, since main split test_endpoints.py;
• its changelog entry is a fragment, since main replaced changelog_entry.yaml;
• comments and one test name that justified keeping x and y for older apps now describe the removal instead (test_records_from_older_executors_keep_their_map_position is now test_records_with_a_map_position_keep_it).

Overlap with #976

#976 adds an identical _is_none helper elsewhere in macro_output.py, and the same pydantic>=2.12 lock changes. Trial merges show that merging this PR, #919 and #924 before #976 leaves one helper and the right pydantic floor, with no conflicts in the contract. Merging #976 first would leave two _is_none definitions until #924 removes this one.

Deployment code and environment variables

No workflow, Dockerfile, Modal app definition or Terraform changes. The Modal images change only through the contract code and the one-line lock changes. No environment variable is added, removed or read differently.

Testing

Written red-green: the tests were written first and seen failing against main's code, then the code was written, and both are committed together.

• Contract: 8 of the 10 new test cases failed on Field required for x and y, or because a null x or y was written to the dump. The other 2 check that records with x and y keep them, which already held.
• Gateway: /jobs/{job_id} returned 500 for a breakdown and a national result whose records lack x and y, with "28 validation errors for JobStatusResponse". The case that sends them as null already passed, because the route drops null fields.
• Generated client: after ./scripts/generate-clients.sh on main's code, the client didn't read a breakdown without x and y as a LocalAreaBreakdownOutput.

Commands run locally, on the new tests only:

• uv run --frozen --all-extras pytest on the new tests in test_macro_output_uk_geography.py and test_local_area_breakdown.py (contract): 10 passed.
• uv run --frozen --extra test pytest on the two new tests in test_simulation_job_status.py, and test_openapi_golden.py (gateway): 4 passed.
• ./scripts/generate-clients.sh, then uv run --frozen pytest on test_generated_client_reads_area_records_without_x_and_y (apis-integ): passed.
• Golden OpenAPI regenerated with python -m policyengine_simulation_gateway.generate_openapi.
• uv lock --check in the six affected projects, ruff 0.15.22 check and format --check on the changed Python files, and .github/scripts/check-changelog-fragment.sh.
• Not run locally: the full test suites, Pyright, the Modal image smoke and the beta integration tests, which CI and the deployment run.

🤖 Generated with Claude Code