Skip to content

Port the CGT structure, salary sacrifice, and student loan stages onto the UK spine at FRS 2024-25 - #740

Merged
juaristi22 merged 5 commits into
mainfrom
uk-cgt-salsac-student-loans-684
Aug 22, 2026
Merged

Port the CGT structure, salary sacrifice, and student loan stages onto the UK spine at FRS 2024-25#740
juaristi22 merged 5 commits into
mainfrom
uk-cgt-salsac-student-loans-684

Conversation

@juaristi22

@juaristi22 juaristi22 commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Closes #684. Workstream E8 of the UK-migration master epic #665, child of #145; rebased onto main at 2aa9679 after #733 merged (digest pins re-derived over the post-#733/#735 union). Targets FRS 2024-25 throughout: build period 2024, calibration year 2025 from the release object, donor arithmetic at the Table 2.1a 2023-24-provisional surface.

What moved

Five manifest stages insert contiguously between hmrc_spi_income_spine and the certified pair (roster 21 → 26, both manifest projections, certified pair pinned at [-2:]):

  1. cgt_incidence_clone — every household (FRS and SPI-synthetic) doubles via the new generic clone_records op: originals no-gains, clones has-gains (household_is_capital_gains_clone), equal mass split w/2 at importance kind with the exact-total bit-correction discipline, MassChangeRecord declared_factor 1.0. The clone household's oldest adult (16+, person_id tiebreak) draws a prior amount from the Advani–Summers percentile surface (linear spline, ext=0, negative draws kept as loss-makers) at an identity-keyed uniform (seed 0). The prior exists because the merged redraw selects gainers as capital_gains > 0, ranks largest-first, and caps the sub-AEA remainder — the prior supplies the gainer set, ordering, sub-AEA survivors, and pass-through losses, exactly as the certified artifact feeds the stage today.
  2. cgt_band_donors — 270 = 30 × 9 donor households (household_is_cgt_band_donor), seeded draw (seed 1) over id-sorted candidates p ∝ A&S incidence at the oldest adult's income, bands positional, initial weights = published band taxpayers / 30 (2,633.33 … 66.67) — never zero (the 18× CI-blowup rationale carried in-manifest); band mean gain on the donor's oldest adult; adds exactly 275,000 weighted taxpayers.
  3. hmrc_cgt_gains_spine — the merged Impute capital gains amounts from the published joint distribution #560/Wire the capital gains family required-at-build with its fence in-manifest #676 Table 3 amounts redraw reused unchanged on the spine path (same ODS pin, seed_base 552 mixed with build period, fence calibration_permitted: false / 76 facts per UK capital gains: imputed distribution has no top tail, missing 55% of CGT #552). The certified-line wrapper cgt_source_stages.json and the national-driver injection stay untouched until Assemble the microcosm-built UK spine: whole-spine parity and swap acceptance #686 — the E7 spine-vs-certified naming precedent.
  4. salary_sacrifice — weighted QRF (seed 42) trained on the FRS survey channel's asked subset only (216 yes / 3,743 no at 2024-25; clones, donors, SPI rows excluded from training), predictions clamp ≥ 0, asked rows byte-preserved; then convert_donors_to_target_stock: shortfall vs the declared 5.4m staging target, donor pool = employee-pension contributors without SS, rate = min(0.5, shortfall/pool mass), identity-keyed selection (seed 2024), full pension moved and source zeroed, headcount receipt into the build sidecar.
  5. student_loans — cohort rules at calibration year 2025 (start year = year − age + 18; repayers → PLAN_1 < 2012 ≤ PLAN_2 < 2023 ≤ PLAN_5, not country-gated), then top_up_to_stock PLAN_5 before PLAN_2 against the SLC liable stocks (10,000 / 8,940,000 at 2025), eligibility England ∧ TERTIARY ∧ age band ∧ cohort band, identity-keyed per-plan salts (seed 42). PLAN_4 never imputed — documented UK-specific gap (the FRS cannot identify the Scottish product; no Scottish liable stock exists).

Supporting surfaces: 13 op kinds registered (the 7 existing CGT kinds + 6 new); StagePlan gains explicit declared rewrites (one-producer check stands otherwise); mass_change_semantics on family coverage so the donor stage's intentional mass increase is validated, not exempted; family coverage closes the capital_gains and student_loan_plan producers; student_loan_plan enum-domain gate (the BRMA evaluator generalized); export surface gains household_is_cgt_band_donor; spine driver wires --cgt-ods, harvests every new seed, and persists the four stages' executed-effect receipts as sidecar stage_evidence.

Committed cited resources

  • hmrc_cgt_size_bands.json — all 13 Table 2.1a rows (the ≥ £12,300 filter stays an op parameter); publisher ODS sha-pinned (69645611…b22491, 10,658 B, retrieved 2026-08-21, content-verified against the extraction) with mapped_build_period: 2024 / period_mapping: latest_published_tax_year per the Retarget the UK build to FRS 2024-25: re-pin raw vintages, regenerate parity instruments, re-measure gate baselines #723 convention.
  • advani_summers_capital_gains_distribution.json — the 61-band incidence + percentile surface, byte-faithful to the incumbent CSV. Citation established (the incumbent had none): Advani & Summers (May 2020), "Capital Gains and UK Inequality", CAGE WP 465 — corroborated: the paper's "~15% of the top 1% have gains" = p99 0.1508; "next percentile 7%" = p98 0.0710. Row 30's p95 is corrected 15,190 → 151,900, verified against WP 465 Table A1 (p. 39) — all 61 rows were checked value-for-value against the paper and the dropped digit was the single mismatch (fix-and-sign; the incumbent's value fabricated loss-makers for band-69 draws above q≈0.962). The prior-draw loader now fails closed on any non-monotone quantile row.
  • salary_sacrifice_anchor.json — the HMRC 7.7m total-users anchor (3.3m above / 4.3m below £2k), the documented 0.7013 staging ratio, and the 5.4m derived stage target.
  • slc_liable_stocks.json — the full Table 6a liable + above-threshold series 2025–2030, chronicle_candidate: true, values equal to the chronicle package slc-student-loan-borrower-forecasts-england-2025 (period note: chronicle stores AY opening year 2024 = our calendar 2025). Consumed as a stage-internal anchor: calibration moves weights and cannot create the ~5m below-threshold plan-holders the FRS never observes — the stage creates the support; calibration later binds the chronicle facts against it.
  • cgt_band_donor_support_bounds.json — published-band intervals, deliberately not wired into the terminal support gate: post-redraw amounts move within Table 3's bands, so the Table 2.1a bounds would be wrong at the terminal surface; the resource documents the inapplicability (see adjudications).

Licensed acceptance (all green; evidence data/ukds/acceptance/e8/)

  • Twin f100 builds payload-identical (twin_payload_compare.json, exit 0) on the full spine roster; the sidecar-evidence driver patch verified payload-inert against the same artifact.
  • Record identity: household 52,846 exactly = (16,288 + 10,000) × 2 + 270; donor mass +275,000 exactly. Person 113,649 / benunit 61,211 vs the reference's 113,617 / 61,223 — donor-composition RNG outcome (different 270 donor households carry different rosters), signed below.
  • e8_identity_receipt PASS: clone-pair structure, donor selection recomputed from the committed resources in original and permuted row order, student_loan_plan recomputed in full — all match the stored artifact.
  • Share screen: capital_gains 0.233684 vs 0.23375 (−0.00007 — the structure-determined share reproduces); student_loan_plan closes at 1.0; pension_contributions_via_salary_sacrifice 0.0796 vs 0.0831 (within tolerance — the E8-pending divergence from Retarget the UK build to FRS 2024-25: re-pin raw vintages, regenerate parity instruments, re-measure gate baselines #733's screen closes); employee_pension_contributions −0.0488 beyond ±0.02, signed below.
  • Incidence-delta instrument: incumbent torch-blend has-gains shares span 0.023–0.477 per A&S band (mean |deviation from 0.5| = 0.428) vs our structural 0.5 — the adjudicated equal-split difference, measured.
  • Receipts: salsac conversion rate 0.166 (uncapped), 4,950 rows converted, post-headcount 5.479M ≈ the 5.4M target; PLAN_5 top-up lands 9,988 ≈ 10,000; PLAN_2 saturates its eligible pool (rate 1.0) — structurally identical in the incumbent.

Signed differences

  1. Equal mass split replacing the torch blend (adjudicated U2) — with the honesty note that only the two aggregate CGT facts calibrate today; the 76 band facts remain fenced pending a separately reviewed target profile (UK capital gains: imputed distribution has no top tail, missing 55% of CGT #552), so aggregate calibration alone blends until promotion.
  2. Table 3 redraw vs the A&S spline amounts (merged Impute capital gains amounts from the published joint distribution #560, restated) — the A&S prior retains its ranking / sub-AEA / loss-maker roles.
  3. Donor-selection and identity-keyed draw streams (standing E4 RNG class) — person +32 / benunit −12 vs the reference; per-plan student-loan streams decoupled by salts; seeded QRF vs the incumbent's unseeded (U14).
  4. Salsac conversion depth (U14 realization) — our seeded QRF lands pre-conversion SS at 2.44M so the declared 5.4M target binds through conversion (zeroing 4,950 donors' employee pension, hence −0.0488); the incumbent's unseeded QRF alone exceeded the target (7.91M on calibrated weights), leaving its conversion inert and its employee-pension share at the raw level. Weighted-distributional comparison binds at Assemble the microcosm-built UK spine: whole-spine parity and swap acceptance #686 under WS-D.
  5. EDUCQUAL-inherited TERTIARY gate (E3) — corrected map gives TERTIARY = 35.2% of adults vs the incumbent's inverted-map 10.6% (2.83×; the corrected figure matches ONS); plan-2 eligible pools land nearly equal under the age gates, plan-5 pools differ 2.8×.
  6. Prior-band income proxy — component-sum total income (no allowance subtraction) vs the incumbent's engine total_income; carrier rule 16+ (FRS adult convention, in-manifest) vs the engine's 18+ is_adult — divergent only for households headed by a 16–17-year-old.

Corrections (recorded on #684 at implementation start)

The 780-donor figure was the 2023-artifact era; at 2024-25 it is 270 = 30 × 9 (and the incumbent's own history went 2,700 → 270, never 780). The "5.4m OBR anchor" is a deliberate ~70% staging fraction of HMRC's 7.7m anchor (the obr/ naming is a misattribution). The torch blend is deterministic — the drop stands on the torch dependency and its loss bug (clone half modeled as (1−s)·s·w in the loss vs (1−s)·w applied).

Two-arm verification (#730/#684 rule)

Every declared parameter carries a drift assert binding the manifest to reviewed code constants (negative-tested per parameter), and an executed-effect receipt: clone mass split → half-mass evidence + identity receipt; donor count/bands/weights → recomputed invariants + per-band sidecar receipt; conversion target/cap → headcount receipt (cap_bound flag); cohort rules/stocks → per-plan receipts; seeds → twin-build determinism (the standing absence statement). The rule itself is documented at the spi_spine reviewed-constants block.

UK-specificity note

UK-specific: the Table 2.1a band surface, the A&S incidence table, the SLC plan rules and PLAN_4 gap, the salsac SALSAC-domain semantics. Generic: clone_records, stack_band_donor_households, top_up_to_stock, convert_donors_to_target_stock, the enum-domain gate generalization, rewrites, and mass_change_semantics.

For María's adjudication

  1. Donor support bounds: accept the documented inapplicability at the terminal surface, or specify a stack-stage-scoped support check.
  2. Record-identity person/benunit deltas (+32/−12): sign as the donor-composition RNG class.
  3. A&S citation (CAGE WP 465) as the resource's established provenance — now corroborated by the full-table verification above.

🤖 Generated with Claude Code

@juaristi22
juaristi22 force-pushed the uk-frs-2024-25-retarget-723 branch from 79f5993 to d70ea39 Compare August 22, 2026 00:20
juaristi22 and others added 3 commits August 22, 2026 10:25
…loans

Five manifest stages between the SPI channel and the certified pair
(roster 21 -> 26, both manifest projections): cgt_incidence_clone
(clone_records equal mass split w/2 + Advani-Summers prior amounts on the
oldest adult, identity-keyed), cgt_band_donors (270 = 30 x 9 HMRC Table
2.1a bands at band-exact never-zero weights, seed 1), hmrc_cgt_gains_spine
(the merged #560 Table 3 redraw reused unchanged on the spine path; the
certified-line wrapper and national-driver injection stay untouched until
seed 42, + conversion to the 5.4m staging target, identity-keyed seed
2024), and student_loans (cohort rules at calibration year 2025 + PLAN_5-
first top-up to SLC liable stocks, identity-keyed seed 42; PLAN_4 never
imputed, documented).

Thirteen operation kinds registered; five committed cited resources
(Table 2.1a with publisher-ODS sha pin, A&S CAGE WP 465 distribution with
the row-30 anomaly documented, HMRC 7.7m salary-sacrifice anchor with the
0.70 staging ratio, SLC Table 6a liable stocks flagged chronicle_candidate,
donor support bounds recorded as inapplicable-at-terminal for review);
per-stage reviewed-constants drift asserts and executed-effect receipts
per the #730/#684 two-arm rule, now documented at the spi_spine
reviewed-constants block; StagePlan gains explicit declared rewrites;
family coverage closes the capital_gains and student_loan_plan producers;
student_loan_plan enum-domain gate (BRMA generalization); export surface
gains household_is_cgt_band_donor; spine driver wires --cgt-ods and
harvests every new seed into the build sidecar.

Closes #684.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…car evidence

e8_identity_receipt in tools/verify_uk_identity_stability.py receipts the
clone-pair structure, recomputes the band-donor selection from the
committed resources in original and permuted row order, and recomputes
student_loan_plan in full against the stored column; the A&S priors, the
Table 3 redraw draws, and the salary-sacrifice QRF and conversion are
scoped to twin-build determinism. The spine driver persists the four E8
transforms' executed-effect receipts into the build sidecar as
stage_evidence (#730/#684 two-arm rule, arm 2).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ed-world CGT asserts, stage-bound mass receipts

Five findings from the pre-push adversarial round, all dispositioned:

1. Gate-battery cross-shard triplet re-pinned in both mirrors from the live
   producer over the post-#733/#735 union (policy 5cb072a0, gates c5123517,
   fingerprint 23cf63b6) - the rebase had left main's pre-E8-gate values.
2. The Advani-Summers percentile-69 p95 corrected 15,190 -> 151,900,
   verified against CAGE WP 465 Table A1 (p. 39); all 61 rows checked
   value-for-value, single mismatch confirmed as the incumbent's dropped
   digit - fix-and-sign replaces replicate-and-document. The prior-draw
   loader now fails closed on any non-monotone quantile row.
3. Salary-sacrifice and student-loan receipts gain expected-vs-realized
   fields (expected mass = rate x pool, realization deviation), quantifying
   the Bernoulli realization the port-faithful mechanism accepts.
4. The CGT spine drift assert now binds every operation's full declared
   parameter mapping closed-world (extra keys rejected), not a subset.
5. Salary-sacrifice and student-loans declare stage-named mass-conservation
   reasons in both manifests (schema extended), emit matching
   MassChangeRecords, and the regenerated coverage manifest binds each
   family to its own receipt instead of the generic fallback (pre-E8
   families keep the fallback - follow-up noted).

Bundle spec sha re-pinned (17c820bc) for the manifest edits; gate triplet
unchanged by them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@juaristi22
juaristi22 force-pushed the uk-cgt-salsac-student-loans-684 branch from b83e334 to 45faff0 Compare August 22, 2026 10:22
@juaristi22
juaristi22 changed the base branch from uk-frs-2024-25-retarget-723 to main August 22, 2026 10:23
@juaristi22
juaristi22 marked this pull request as ready for review August 22, 2026 13:37
@juaristi22 juaristi22 closed this Aug 22, 2026
@juaristi22 juaristi22 reopened this Aug 22, 2026
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Adversarial review (2026-08-22)

A Codex adversarial round on the rebased branch returned 3 high + 2 medium findings, all dispositioned before this push (one CI round): (1) the cross-shard gate-battery triplet re-pinned from the live producer — the rebase had kept main's pre-E8-gate values; (2) the A&S row-30 correction above; (3) expected-vs-realized fields added to the salsac and per-plan receipts (measured: salsac +2.7%, PLAN_5 −1.4%, PLAN_2 exactly 0 — pool-saturated and deterministic), with the Bernoulli top-up mechanism itself kept port-faithful; (4) the CGT spine drift assert extended to closed-world full-mapping equality per operation (extra keys rejected); (5) salary-sacrifice and student-loans now declare stage-named mass-conservation reasons in both manifests, emit matching MassChangeRecords, and the coverage manifest binds each E8 family to its own receipt (the pre-E8 families' generic fallback is a noted follow-up). Licensed twins rebuilt at the disposition commit: payload-identical, identity receipt PASS, all screens unchanged.

Implemented by Codex under /codex-implement-plan (6 recorded deviations, all reviewed and kept: the 13th op kind; StagePlan.rewrites; mass_change_semantics + canonical-family inheritance; cross-shard gate digest re-cuts; ODS-sha deferral — closed in review with the fetched publisher sha; a venv-launcher workaround). Claude review added the ODS provenance pin, the changelog slug, the e8_identity_receipt instrument, and the sidecar stage_evidence patch.

Fixes the nondeterministic main-CI red first seen on the FRS 2024-25
retarget merge run (test_reform_materialization_builds_one_engine_system
_per_family asserting 0 > 1000): variable-module names are keyed by
id(system), so in a warm suite process CPython can hand a fresh
CountryTaxBenefitSystem a dead prior system's recycled address,
re-registering its ~5,600 modules under already-existing names and
zeroing the measured delta. No in-process counting survives id
recycling, so the measurement now runs end-to-end in a subprocess whose
import state is virgin by construction; the pytest test asserts on the
child's outcome and surfaces its output on failure. Not an E8 surface -
fixed-and-signed in the passing PR per the standing precedent (the WAS
bridge-donor pin fix rode the retarget PR the same way).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Main-CI red addressed in this PR (e418f756)

Main's Tests run on the #733 merge commit (run 32562024198) failed on test_us_fiscal_refresh_memory.py::test_reform_materialization_builds_one_engine_system_per_family with assert 0 > 1000 on the leak-metric liveness check — not a regression from #733's diff (the identical tree passed its PR CI; the file passes locally with the same policyengine-us==1.764.6 CI installed; nothing in #733 or this PR touches that surface).

Root cause: the canary counts sys.modules entries named by id(system). In a warm suite process, CPython can hand a fresh CountryTaxBenefitSystem a dead prior system's recycled address, re-registering its ~5,600 variable modules under already-existing names — the measured delta reads 0 and the liveness assert fails. Allocator-state-dependent, hence green-then-red on the same tree.

Fix (fixed-and-signed here, the #733 WAS-pin precedent): no in-process counting survives id recycling, so the measurement now runs end-to-end in a fresh interpreter (subprocess), where the baseline is virgin by construction; the pytest wrapper asserts on the child's outcome. Verified: the file's 8 tests pass, and the child measures cleanly even from a parent deliberately pre-warmed with an engine build — the exact condition that killed the CI run. Once this PR merges, main inherits the fix.

…tinct CGT receipt identities

Two findings from the pre-merge adversarial round, both dispositioned:

1. Closed-world coverage was only on the spine amounts stage. Every E8
   stage assert now binds the full declared operation sequence and each
   operation's complete parameter payload (extra keys, missing keys, value
   drift, and extra or reordered operations all fail by name) via a shared
   ordered-sequence helper; the donor stage's previously-unbound propensity
   declaration is covered, with mutation tests for propensity, extra keys,
   and extra operations on all four stages.

2. The certified and spine CGT families shared one mass-conservation
   reason, so a single record satisfied both. The reused runtime gains an
   additive mass_change_reason parameter (certified default unchanged); the
   spine projection declares and emits its own spine-specific reason in
   both manifests, and the coverage generator now rejects duplicate
   stage-declared reasons outright. The five pre-E8 families sharing the
   generic fallback are a standing follow-up, unchanged here.

Bundle spec sha re-pinned (1f163cbf) for the manifest edit; gate triplet
unmoved.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

Second adversarial round dispositioned (718c088e) — nothing blocking from the review side

The pre-merge adversarial round returned 2 high findings, both bypass gaps in the first round's own dispositions, both fixed:

  1. Closed-world asserts everywhere, not just the spine amounts stage. Every E8 stage assert now binds the full declared operation sequence and each operation's complete parameter payload — value drift, missing keys, extra keys, and extra/reordered operations all fail by name (the donor stage's propensity declaration was previously unbound; the reviewer's exact mutation now fails). Mutation tests cover propensity, extra-key injection, and extra-operation appends on all four stages.
  2. Distinct CGT receipt identities. The certified hmrc_cgt_gains and spine hmrc_cgt_gains_spine families shared one mass-conservation reason, so a single record satisfied both. The reused runtime gains an additive mass_change_reason parameter (certified default byte-identical); the spine projection declares and emits its own reason in both manifests, and the coverage generator now rejects duplicate stage-declared reasons outright. The five pre-E8 families sharing the generic E5 fallback pre-date this PR (verified against main's manifest) and remain the noted follow-up.

The reviewer independently confirmed the US canary subprocess fix works and that the gate triplet and bundle digests match their live producers.

Licensed evidence re-run at 718c088e: twin builds payload-identical, e8 identity receipt PASS, record identity 52,846 exact, all screen values unchanged.

Merge readiness: with this round clean, the remaining open items are the three adjudications in the PR body (donor support-bounds inapplicability, the person/benunit +32/−12 donor-composition deltas, the A&S CAGE WP 465 citation) — all sign-offs rather than changes. CI on 718c088e is the last gate.

@juaristi22
juaristi22 merged commit 055dcfa into main Aug 22, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Port capital gains structure, salary sacrifice, and student loan stages

1 participant