Repository navigation
Refuse new policies whose data names paths that are not parameters - #4123
Merged
Merged
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #4123 +/- ##
==========================================
+ Coverage 88.13% 88.15% +0.02%
==========================================
Files 273 274 +1
Lines 16299 16393 +94
Branches 2668 2689 +21
==========================================
+ Hits 14365 14452 +87
- Misses 1268 1271 +3
- Partials 666 670 +4 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
A new v1 policy may name only parameters that the running country model can set a value on. These tests cover unknown paths, parameter groups, scales, scale brackets and missing contributed parameters for the US and the UK, plus valid scale bracket and contributed parameters, and check that re-saving an identical stored policy still returns it. They also cover the household calculation check, which currently accepts parameter groups because the metadata lists them as parameterNode entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
POST /<country_id>/policy now checks every key in data when the request would store a new policy. Each key must name a Parameter in the country's loaded tax-benefit system, the one the Public API uses for its metadata and household calculations, so scale bracket values and contributed parameters the model defines are accepted. Unknown paths, parameter groups, scales and scale brackets get a 400 whose message and errors list every refused path. Re-saving a policy identical to a stored one still returns it with a 200, without checking its paths. The household calculation check tested only whether a policy key appears in the metadata's parameters, which also lists groups, scales and brackets as parameterNode entries. It now accepts only entries of type parameter, so a group gets the existing 400 instead of reaching the model and failing with a 500. The metadata is unchanged. Fixes #4122 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test created a policy that sets sample_parameter, which the US model does not define, so policy creation now refuses it with a 400. It sets gov.contrib.states.mt.ctc.in_effect instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk
force-pushed
the
policy-path-validation
branch
from
October 29, 2026 20:13
6392e3d to
953f6bb
Compare
UnsettableParameterPathsError now returns the 400 response's errors entries itself, so the policy route passes them through instead of converting each one. A model parameter lookup that returns None, for a country without a loaded model, now leaves a new policy no parameters to set, so every path is refused. Every country the policy service accepts has a loaded model, so this replaces a branch that could not be reached in production, and a service test covers it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4122
Summary
POST /<country_id>/policyis the only route that creates a v1 policy. When a request would store a new policy (the 201 case),PolicyService.set_policynow checks every key indataagainstPolicyEngineCountry.parameter_names(added by Report which parameters a policy sets that the current model no longer has #4039): the names of everyParameterin the country's loaded tax-benefit system, the same system the Public API uses for its metadata and household calculations. The route reads them through Report which parameters a policy sets that the current model no longer has #4039'sloaded_model_parameter_names, and the comparison isparameters_missing_from_model, the one the v2 copy andmissing_parametersuse. Scale bracket values such asgov.irs.credits.ctc.amount.base[0].amountand contributed parameters the model defines are accepted. Unknown paths, parameter groups, scales and scale brackets get a 400 whose message anderrorslist every refused path, sorted. Nothing is stored.find_unrecognized_policy_inputstested only whether a policy key appears in the metadata'sparameters, which also lists every group, scale and bracket undergovas"type": "parameterNode". A group therefore passed the check, reached the model, and failed with a 500 ('ParameterNode' object has no attribute 'values_list'). The check now accepts only entries of typeparameter, so a group gets the existing 400./metadatais unchanged./<country_id>/policy(82fe6ed2c0735fa9) #4098: those policies, such as 98762 (gov.irs.credits.ctc.amount), 98768 (gov.contrib.states.mt.hb1.in_effect) and 98770 (gov.x), can no longer be created.noqasuppressions ortype: ignorecomments are added.Example 400:
{ "status": "error", "message": "Policy data contains keys that are not parameters in policyengine-us 2.2.1: gov.contrib.states.mt.hb1.in_effect, gov.irs.credits.ctc.amount, gov.x", "result": null, "errors": [ { "type": "policy_parameter", "name": "gov.contrib.states.mt.hb1.in_effect", "path": "data.gov.contrib.states.mt.hb1.in_effect", "message": "Unrecognized policy parameter `gov.contrib.states.mt.hb1.in_effect` at `data.gov.contrib.states.mt.hb1.in_effect`." } ] }(
errorsholds one entry per refused path; the other two are omitted here.)Testing
gov.x, a parameter group, scale and scale bracket, a missinggov.contribpath, and valid scale bracket and contributed paths, for both the US and the UK, through the policy create route and the household calculate route, plus service tests with an injected parameter catalog.make testpassed 3909 tests with 0 failures. After the rebase, the test files this pull request adds or changes pass: 146 passed.python scripts/run_quality_guards.py: 5 of 5 guards passed.python scripts/export_migration_contracts.pyleft the generated files unchanged.ruff format --check .(746 files already formatted),ruff check ., andmypy(169 source files, no issues) pass. The changelog fragment check findschangelog.d/4122.fixed.md.tests/unit/routes/test_policy_dual_write_routes.pyandtests/to_refactor/python/test_policy.pycreated policies with placeholder paths through the real service; they now use real US parameters. Test metadata fixtures whose parameter entries had notypenow give them"type": "parameter", as the real metadata does.tests/data(Utah, California EITC, Oregon, UK Universal Credit, and the economy test policy) set only settable parameters, so the deployed checks keep working.🤖 Generated with Claude Code