Triage policyengine-us-data backlog for Populace migration

[MaxGhenis]

Context

policyengine-us-data is no longer the live build surface for replacement microdata. Populace supersedes it, so open policyengine-us-data issues/PRs should be triaged into Populace work only when the underlying data-quality, target, export, or release-contract concern is still relevant.

This issue records a pass over the live open PolicyEngine/policyengine-us-data issues and PRs on 2026-06-14, after Populace PRs #44 and #45 merged.

Already covered or mostly covered in Populace

• policyengine-us-data #1156 / PR #1157: microunit tax-unit construction. Populace already delegates US tax units and SPM units to microunit through populace-frame.
• policyengine-us-data #1171 / PR #1171: negative auto_loan_interest from SCF sentinels. Populace already has manifest-declared sentinel handling plus nonnegative source/export gates for auto_loan_interest.
• policyengine-us-data #1105, #1107, #872, Bind SSA SSI age-band recipient counts as ordinary indicator targets (#470) #476, CHIP enrollment targets are concept-mismatched: CMS total_chip_enrollment includes M-CHIP; model materializes separate CHIP only (20 states zero-support) #321, UK: household-wealth imputation stage with cash / S&S ISA split #181, social_security stored input contradicts its four components for ~2k records (decomposition gap) #183, Add proximal L1 solver path #184, Add fiscal diagnostics comparison tool #190, Remove count aggregation from requirements example #192: fiscal/SOI/JCT/state-tax target surface. Populace US calibration does not pin federal income tax or AGI; base undershoots income tax ~18% — add SOI fiscal targets + a macro-realism gate #40 / PR Restore US fiscal calibration targets #44 added federal income tax, JCT tax-expenditure reform contracts, state income tax, SOI rows, and macro-realism gates. Keep remaining target refreshes as future target-vintage work, not old-repo work.
• policyengine-us-data Make zero-valued calibration targets intentionally nonzero-weighted #104, Doctrine: survey-derived statistics (poverty above all) are permanent calibration holdouts #596 and PE-US #8614 follow-up: partnership/S-corp formula aggregate vs leaves. Populace Propagate partnership/S-corp income split (policyengine-us#8614) into the US build #35 / PR Enforce partnership S-corp leaf exports #45 now exports the PE-US leaves and blocks partnership_s_corp_income as a formula-owned compatibility column.
• policyengine-us-data #1165 and PR #1054 concepts: target coverage / target-policy reporting. Populace Use Arch US poverty/nonfiler coverage contract in Populace target gates #29 and Add NIPA and program-admin nonfiler controls to the US target profile #30 already track source-family coverage and hard-target inventory. Copy only the useful fail / warn / diagnostic_only policy concept if Populace needs more granular enforcement.

Copy into existing Populace issues

• Populace Add NIPA and program-admin nonfiler controls to the US target profile #30: program-admin hard targets and nonfiler controls.
  • Copy/source-link policyengine-us-data #1148 / PR #1162 for CHIP enrollment controls.
  • Copy/source-link #1147, Import-entry unit family P1: ledger ingest of official trade margins (#615) #620 / PR #1087, #889, Restore the SIPP donor half of the financial-asset blend (50/50 SIPP+SCF per household) (#374) #510 for ACA marketplace/APTC take-up and state targets.
  • Copy/source-link #1136, Retirement surfaces are owned by the base builder: no release-time refit #557 for Medicaid category/enrollment reconciliation.
  • Copy/source-link #1094, Assign US congressional districts from Ledger facts #188, Add opt-in ACS congressional district age targets #189, [corrected] populace_us_2024 itemized-deduction report retracted (measurement artifact); residual = state-level SOI income targets #122, #1021 for SSI/OASDI/disability support and take-up.
  • Copy/source-link Migrate the US stacked driver onto microcosm.build.logbook_adoption #689, Standing per-component repeal-revenue validation surface (#298) #535, Port Section 199A simulation and audit processed PUF inputs #532 for LIHEAP/TANF/WIC/SNAP/take-up details where they map to source-backed targets.
• Populace Track SPM-specific input gaps: housing, WIC, school meals, child support, workers comp, and expense deductions #32/Add a Populace US poverty/SPM diagnostic gate for resource distribution and negative-income tails #33: SPM/resource diagnostics.
  • Copy/source-link #1113, WS-C deferred adjudications and the reconciliation layer #736, Retarget the UK build to FRS 2024-25: re-pin raw vintages, regenerate parity instruments, re-measure gate baselines #733, Complete #622: compile the UK target surface from Ledger facts at the 2024-25 line #735, Add Logbook family model and typed exact-k cardinality #739, Port the CGT structure, salary sacrifice, and student loan stages onto the UK spine at FRS 2024-25 #740, Repair the stale lock: microcosm-frame's declared uk extra #732, Treat PE-US aggregate outputs as formula-owned #182, Refit crushes keogh carrier weights to $6.1M of $148.97M — smoke floor unreachable on the sparse arm (attempt 15) #445, Bind every blocking gate verdict into release certification: waived gates vanish from manifests and publication accepts any nonempty gates mapping #381 for housing assistance, SPM expenses, child support, CE/MEPS/AHS-style source gaps and validation-only diagnostics.
• Populace Populate remaining US tax-input and reported-observation layers #38: remaining US tax-input and reported-observation layers.
  • Copy/source-link #1126, #1127 / PR #1128, #900 for capital-gains basis / holding period / SOCA-style imputation.
  • Copy/source-link qualified_tuition_expenses essentially unimputed (641 records, $4.5B) — education credits understated ≥40% #253, Run ACA source refresh in fiscal release builder #85, Country-agnostic gate battery core: phased evaluation, unconditional reporting, per-country gate selection (#611) #619, populace-us-2024 c86a631 (2026-06-19) build stores weekly_hours_worked_before_lsr as constant 40 — SNAP work-requirement analyses silently return zero effect #613 for section 179, Alaska PFD separation, liability pruning, and top-tail AGI support.
  • Copy/source-link #1160, Add calibration warm-start and target-frame checkpoints #227, Carve the PUF donor's E19200 lineage to the mortgage-only concept (#515 interim) #525, Two-signal FLSA overtime incidence: usual-hours leg joins the reference-week snapshot (#451 item 4 residual) #526, Screen aggregate-scale mortgage outlier rows out of the PUF donor (#516) #527, ORG hourly-wage QRF imputes sub-minimum wages for part-year workers (donor income is annualized-weekly; recipient income is actual-annual) #529 for tips and Stage-1 source quality failures.
  • Copy/source-link Close the schema-3 terminal-report path for non-grandfathered exact-k UK releases #712, Score legacy flat PE H5s #219, Add direct CHIP enrollment targets to the US fiscal surface #139, Allow staging telemetry repo config from environment #138, UK: surface cash ISA and stocks & shares ISA holdings in the wealth imputation #180, Remove aggregation method from Ledger target selectors #193, Score OBBBA reform validation stacked (match JCT) #185 for TIN/SSN/pay-period/pregnancy/widowship support if those remain PE-US input needs.
• Populace Drop raw-source and build-intermediate columns from the published populace-us artifact #27/Fix in_nyc before enforcing no-formula exports #34: export-surface and formula-owned storage.
  • Copy/source-link Restore the release H5 write that #437 dropped #444, #829, Add Census state SPM poverty-rate backtests (104 out-of-sample validation rows) #348, UK output-area-anchored geography ladder (US block-ladder pattern) #349, #814, farm_income is an unpinned export dimension (109 selected carriers) — parity gate fails Build M attempt 12 #441, Add UK SPI support rows #149 where they identify formula-owned exports, entity-level ID propagation, filing-status, or naming issues still relevant to Populace.

New Populace follow-up issues to consider

• PUF real-half / clone support and release gates.
  • Source: policyengine-us-data #1139, #1158 / PR #1159, PR #1170, Delivery gate can prescribe a provably identical retry when the basis is already true (post-#549/#550 residual) #555, Impute capital gains amounts from the published joint distribution #560, Transfer targets must be engine input leaves: drop formula-owned weeks_worked, fix medicare premiums leaf, add ownership guard (#578 inc 2) #590, ACS transfer predictors accept semantic booleans; adjudicated null handling; full predictor sweep (#578 inc 2) #595, Rowwise local solve surface under the doctrine (#495 increment 6b) #541.
  • Populace should decide whether it needs explicit real-half vs clone-half support diagnostics, a PUF-clone weight-share floor/cap, and smoke tests for PUF-imputed charitable/business/tax-detail variables. This should stay spec-driven: country manifests can declare the support contract, but shared runtime code should implement the operation.
• Geography and local-area support.
  • Source: #1143, #1144, #1145, #1146, PUF stage manifest declares a support_clip op that no execution path honors (dead donor_realized contract) #482, ESI employer premium contributions as a first-class, validated resource component #454, Materialize Marketplace selected-plan benchmark ratios and metal-level diagnostics #75, Bind SOI Table 2.1 mortgage targets to the capped deductible concept (#511 target-side leg) #517, SSA SSI recipients by age band: mint as ledger facts, bind as registry count targets #470, Restore strike_benefits from the OI_VAL decomposition (#591 — ECPS parity) #598, PR #1088.
  • Populace should track boundary vintage, district geojson provenance, county/rating-area assignment, local H5/local-area build contracts, and matrix scaling separately from the old staged pipeline.
• Release manifest / publication contract.
  • Source: Confirm whether count/amount categories should affect US fiscal target loss weights #724, #799 / PR #1167, Use target-period Medicaid facts in the 2024 US fiscal refresh #91, Populate reported TANF, SNAP, and WIC enrollment from CPS admin-linkage fields (#591) #600, Logbook: record requested and realized exact-k cardinality #641, The stacked spine: one origin-labeled spine, banked gap-fill, null-vs-zero doctrine, by-origin battery (#578 revision pilot) #616, Add the L2 sweep harness over exported production calibration inputs #310, Tracked publish wrapper to wire release Slack alerts #179, Port UK compact source stages and 2024-25 refresh from uk-data #145, Cache US target reform materialization #201, Fill missing current CD targets from state totals #209, PR #1071, PR #1077.
  • Populace needs the useful release-manifest ideas, compatible model version metadata, diagnostics summaries, and published-run indices, but not the old eCPS Stage 5 implementation shape.
• Wealth/property/asset imputation.
  • Source: #1079, #1080, #1081 / PR #1085, Add coverage-universe targets for uninsured and private coverage #230, Calibrate the undocumented population count as a Ledger fact (weights lane follow-up to #266) #267, Polish follow-ups from the target-aging review (#280) #281, #835, Populate passive_partnership_s_corp_income: SCF non-active-business shape, Form 8960 level #734.
  • Populace already declares scf_primary_residence_value; still verify whether PE-US-facing primary_residence_value, home equity, vehicle variables, auto-loan eligibility, net-worth decomposition, and Fed Financial Accounts benchmark replacement need dedicated Populace tasks.

Move outside the live Populace repo

• policyengine-us-data #1163 / PR #1164: production eCPS baseline resolver and integrity gate. The lesson is important, but incumbent/eCPS comparison should live in PolicyEngine/populace-benchmarks or another benchmark repo, not in live Populace build code.
• Any remaining eCPS/local-baseline promotion comparisons should be benchmark-repo work only. Populace release gates can expose diagnostics; they should not depend on a stale incumbent artifact.

Do not migrate directly

• Microplex-specific issues such as #1142 and populace_us_2024 under-1 geography: CA carries 21.7% of US infants (~1.9x its population share), MI 1.0% (a third of its share) #520.
• L0/national-penalty work such as PR #1166 and PR Net-new UK stages: UC deduction attributes, bus fares, WAS debt #685, except for high-level calibration-policy lessons already captured above. Populace should stay on its current calibration stack and spec-driven country content.
• Old staged-pipeline service-boundary PRs/issues (#1118, #1116, #1104, #1100, #1098, #1086, #1083, #1078, #1074, #1073, #1066, #1049 and related PRs #1119, #1117, #1106, #1101, #1099, #1084, #1082, #1076, #1075, #1051, #1068) unless a specific contract concept is missing in Populace.
• Temporary threshold loosening / broken eCPS build patches (#1160 / PR #1161) except for the underlying target or source-quality causes listed above.

Acceptance criteria

• Existing Populace issues above are updated or linked with the relevant source backlog.
• Any missing Populace issues from the follow-up section are opened or explicitly rejected.
• No old policyengine-us-data, Microplex, L0, or eCPS comparison implementation is copied into live Populace without translating it into Populace's spec-driven architecture.
• Benchmark-only eCPS comparison work is tracked outside live Populace.

[MaxGhenis]

Follow-up trackers opened from this audit:

• Port US PUF real-half and clone support diagnostics #47 for PUF real-half / clone support diagnostics and release gates.
• Track US geography and local-area source support #48 for geography, local-area support, boundary vintage, and rating-area/county assignment.
• Verify US wealth, property, and asset imputation backlog #49 for wealth, property, and asset imputation backlog verification.
• Port release manifest and publication diagnostics contracts #50 for release manifest, compatible model version metadata, and publication diagnostics contracts.

Existing tracker cross-links added to #27, #29, #30, #32, #34, and #38.

[MaxGhenis]

2026-06-27 delta while working toward #204:

• New policyengine-us-data #1172 / PR #1173 is old-stack release-orchestration cleanup. It does not need a Populace data migration, but Replace all remaining policyengine-us-data artifacts with Populace #204 should preserve the useful contract: .py owns stack inclusion, while Populace owns data release artifacts and manifests.
• Old PE-US-data PRs still open around eCPS thresholds, local baseline comparison, old Stage 1-5 service boundaries, publication runners, and bundle contracts should not be merged as a path to replacement. Translate only the durable contract into Populace specs/diagnostics/release tooling.
• The remaining content-bearing old issues still map to existing Populace trackers: PUF clone/support (Port US PUF real-half and clone support diagnostics #47), local geography/CDs (Track US geography and local-area source support #48), wealth/property (Verify US wealth, property, and asset imputation backlog #49), release manifest/publication (Port release manifest and publication diagnostics contracts #50/Replace all remaining policyengine-us-data artifacts with Populace #204), fiscal/source target coverage (Add NIPA and program-admin nonfiler controls to the US target profile #30/Populate remaining US tax-input and reported-observation layers #38/Consume Ledger target profiles instead of carrying target values in Populace #160), health/ACA/CHIP (Jointly calibrate ACA take-up to CMS APTC people and SOI PTC returns #168-Add Medicaid and CHIP eligibility-to-enrollment diagnostics #170, Materialize Marketplace selected-plan benchmark ratios and metal-level diagnostics #75/Migrate ACA health-input gates after PE-US take-up contract changes #76/Track health input readiness for upcoming PolicyEngine-US model improvements #98), and formula-constrained leaf imputation (Use formula-constrained leaf imputation for deduction inputs #186).
• Active replacement build: a current-main full CD target-surface rerun is queued with CD rows diagnostic-only for small releases; results should feed Replace all remaining policyengine-us-data artifacts with Populace #204 acceptance criteria before any .py default move.

[MaxGhenis]

Completed 2026-07-02: the full policyengine-us-data backlog (146 issues, 27 PRs) was triaged and closed using this issue's mapping as the authoritative source, the README archive banner merged (policyengine-us-data#1177), and the repository is now archived. Per-item mapping recorded on policyengine-us-data#1175.