Omit population-ranked variables from illustrative household results

[vahid-ahmadi]

Part of #250 (§3, the household_income_decile regression item).

The bug

#250 asks for a regression test documenting that household_income_decile is ranked against the current simulation population, and notes household_income_decile: 10 for a £6,000 single earner "looks wrong". It is worse than implausible — it is constant. An illustrative household simulation contains exactly one household, so any variable defined by that household's position within the simulated population ranks it against itself:

employment_income	income decile	wealth decile	relative poverty BHC	absolute poverty BHC
£3,000	10	10	0	1
£6,000	10	10	0	1
£25,000	10	10	0	0
£200,000	10	10	0	0

(measured on policyengine-uk 2.89.2, year 2026)

These are default output columns, so the model sees them on every household simulation and can report a synthetic household as top-decile and not in relative poverty whatever its income. Relative poverty is the same defect: a single household is its own median, so the flag can never be 1.

The change

• POPULATION_RANKED_HOUSEHOLD_VARIABLES in backend/engine/constants.py names the four affected variables with the reason and the society-wide tool to use instead.
• calculate_household removes them from baseline and reform result blocks and reports them under omitted_population_ranked_variables, so the model is told the value is unavailable and why rather than being left to infer.
• System prompt rule under ANALYTICAL NOTES: do not state or estimate a synthetic household's national decile or relative poverty status; use a society simulation for a distributional position.
• docs/engineering/skills/uk-chat-runtime.md records the boundary, per the AGENTS.md rule that canonical docs move first.

Removal is deterministic in the engine. The prompt rule reinforces it but is not what enforces it.

Absolute poverty (in_poverty_bhc, in_poverty_ahc) uses a fixed threshold and stays available. Society-wide tools are untouched: they simulate the full weighted population, where these variables carry their intended meaning.

Scope judgement for review

#250 names only household_income_decile. I extended it to the wealth decile and both relative poverty flags because they are the same defect with the same user-visible consequence. Happy to narrow to the decile alone if you would rather keep the beta diff minimal.

Verification

• New backend/tests/test_household_population_ranked_variables.py — 6 tests, including one reading the engine directly, before removal, to pin the constant-across-incomes behaviour.
• New tool_contract eval case household_omits_population_ranked_variables and answer case answer_household_decile_not_national (the answer must decline a decile rather than infer one).
• make eval-ai-offline: 118 passed, 0 failed (116 before this branch).
• Full backend suite: 480 passed. The 4 test_modal_app.py preview-cleanup failures reproduce on a clean checkout of main and are unrelated to this change.

🤖 Generated with Claude Code

https://claude.ai/code/session_01W5Zsotd2SND15F9efcePoP

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
policyengine-uk-chat	Ready	Preview	Aug 24, 2026 1:29pm

[github-actions]

Beta preview is ready.

• Frontend: open preview
• Backend: open backend

[codecov]

Codecov Report

❌ Patch coverage is 90.90909% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.07%. Comparing base (536bd06) to head (bb65c6d).

Files with missing lines	Patch %	Lines
backend/engine/households.py	90.47%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #256      +/-   ##
==========================================
+ Coverage   68.00%   68.07%   +0.06%     
==========================================
  Files         109      109              
  Lines        6827     6848      +21     
  Branches     1390     1398       +8     
==========================================
+ Hits         4643     4662      +19     
- Misses       1771     1772       +1     
- Partials      413      414       +1     

Flag	Coverage Δ
backend	81.33% <90.90%> (+0.03%)	⬆️
frontend	33.22% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
backend/engine/constants.py	90.00% <100.00%> (+0.52%)	⬆️
backend/prompts/system.py	100.00% <ø> (ø)
backend/engine/households.py	92.30% <90.47%> (-1.03%)	⬇️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.