You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Parent: #145 (workstream E of the UK-migration epic #665). Successor to #686. Reconciled 2026-08-25 with the #623 calibration-seam plan after the first armed calibration campaign (9 runs on the uk-first-calibrated-run-623-686 build branch).
Scope, and what moved out
This issue owns the spine lane after #747 merges: the defect batch the calibration campaign surfaced (A), the per-step gating regime for the spine build (B), and the W9–W11 retirement plus dead code/markdown cleanup (C).
Moved out to #623 (PR #743): the calibration-only driver. The new production calibration seam (tools/calibrate_uk_national_dataset.py + calibration_seam.py) is the "national driver becomes calibration-only" step — it takes whichever h5 it is given, materializes measures, solves weights under doctrine, and never modifies a data variable. What stays here is the other half: retiring the legacy June path (tools/build_uk_national_dataset.py, with its SPI re-derivation and imputation stages) once the seam is proven, which is spine-lane cleanup per the 2026-08-24 ruling.
Gate: nothing here starts until #747 is reviewed and merged. Every item either modifies the spine build (invalidating #747's licensed evidence) or retires a fallback the comparison depends on.
State at the point of handover
leg
state
L0 build + determinism
complete — 3 rungs, twins payload-identical, record identity exact at 52,846
loss 0.079 · 378/388 bindable · candidate beats incumbent 293–48 on the frozen register · fiscal totals (UC, income tax) still 10–18% low
A. Spine defect remainder
Two of the five landed in #747 rather than waiting for a rebuild here — the review made the case that fixing the spine before the swap is cheaper than certifying it twice:
Spine sidecar self-pin.spine-c.build.json carries the full stage census, artifact/resource pins, seeds and rules-engine version — and no reader joins it today. Downstream consumers should inherit provenance from the sidecar rather than re-deriving it; wire the join and pin the h5 to its sidecar.
Support-channel gaps: slc.repayments.england_postgraduate has zero support on the spine; the top-income (£1m+) channel holds 2 records on both datasets. Each needs a measured support channel or an explicit adjudication that the gap is accepted.
Investigation, not yet a code fix: fiscal totals 10–18% low on the calibrated candidate increasingly looks like spine income distribution rather than target defects. Assessment lane; findings feed back here or into a successor.
B. Per-step gating regime for the spine build
The 2026-08-24 ruling: no imputation or spine gating in the calibration path — the seam runs a calibration-scoped battery only. That is only coherent if the spine build polices itself, stage by stage, the way the US builds do (#327 doctrine):
Per-stage imputation-health gates. Each imputation stage gets a gate consuming its own receipts (support bounds, incidence vs donor, conservation) so a regression is caught at the stage that caused it, not at a whole-spine diff. The signed-differences register stays the swap-decision record; the per-step gates are the standing health checks.
Wire the dormant CGT donor support bounds — or decide they cannot be terminal.cgt_band_donor_support_bounds.json is declared in country_package.json and read by nothing; the other four support-bounds resources (was_wealth, lcfs_consumption, etb_vat, etb_services) are each listed in the uk_support gate's support_bounds_resources and in battery_bindings.py. The CGT one is not an oversight — the resource records why in its own semantic_fit_note: its intervals describe donor-stage initial support, and E8's hmrc_cgt_gains_spine Table-3 redraw moves amounts onto a different band surface afterwards, so "terminal support-gate applicability requires reviewer confirmation".
That makes this the sharpest argument for per-step gating rather than a stray wiring chore. The other four work as terminal gates because nothing rewrites their columns after they clip to donor; CGT is the one stage where a terminal gate provably cannot express the constraint. Two ways out, and the choice is the adjudication: gate the donor stage at stage time, before the redraw, on the bounds as they stand; or derive post-redraw bounds from the Table-3 surface and gate terminally like the others. Either way the resource stops being declared-but-dead. (This also closes the third of the three sign-offs left open on Port the CGT structure, salary sacrifice, and student loan stages onto the UK spine at FRS 2024-25 #740.)
Re-home the family-coverage contract.uk_release_family_build_stages requires 12 required_at_build families; the spine driver builds 10, the June driver 2, so neither driver alone can pass — the W9 seam. Fix by pointing the guard at the spine plan that actually builds the families (the spine's hmrc_spi_income_spine / hmrc_cgt_gains_spine supersede the June driver's two). Never by relaxing the gate, and never by sidecar plumbing — family_coverage status='required_at_build' is an executable contract, not documentation.
Release-cut certification. The microcosm-data release-verification contract expects the full 21-entry battery on a published artifact — that expectation maps to the monolith. Combining spine-side per-step gating + the seam's calibration-scoped battery into a publishable certification is contract evolution; until it lands, seam-built records say shippable: false with the reason.
C. W9–W11 retirement, dead code, markdowns
Retire the legacy June path once the seam reproduces the campaign result (the v9-equivalent live proof in the Calibrate the UK national build from Ledger-backed targets #623 plan): tools/build_uk_national_dataset.py's SPI re-derivation, replace_uk_spi_support_tables's zero-weight-channel refusal, and the CGT redraw all encode the June convention the spine has superseded.
W8 — certified-identity registry entry for the spine artifact. W11 — retire the CGT injection path the incumbent needed.
UK_CANDIDATE_DATASET_NAME → microcosm_uk_2024 (uk_runtime/terminal_gates.py:72, currently populace_uk_2023). Ruling 2026-08-25: the candidate carries the microcosm name, not the legacy populace one — superseding the earlier populace_uk_2024 note; the repo itself has moved to PolicyEngine/microcosm, and the parity-instrument tests already use microcosm_uk_2024.h5 as the canonical candidate filename. Sweep the stray populace_uk_2024* fixture names in test_uk_rowwise_build_driver.py in the same pass. Check UK_REFERENCE_DATASET_NAME alongside: once the spine is the build, "reference" means the previous certified microcosm line, not the enhanced-FRS artifact.
Un-pin the incumbent from microcosm-data/contract.py:219 and uk_runtime/weighted_integrity.py:241 once nothing reads it as an input. The parity reference itself stays — it is the historical comparison instrument and keeping it is how the swap remains auditable.
Dead code: UKPolicyEngineAdapter (zero production callers — the harness rebuilt its capability outboard); the assessment harness tools/calibrate_uk_national_spine.py retires once the seam driver covers it.
Markdown dispositions — a committed .md is a claim nobody re-checks; promote to a gate/test, make regenerable, or archive with a date:
file
disposition
UK_COVERAGE_PROGRESS.md
stale and misleading — baselined on the enhanced-FRS 2023-24 artifact; live content already lives in release_input_coverage_manifest.json. Delete.
experiments/686-uk-spine-comparison-ledger.md
the adjudication packet; make regenerable so it cannot drift from the register beside it
experiments/686-uk-spine-swap-receipts.md
R0–R5 run receipts. Archive as dated history once the swap lands
experiments/630-uk-gate-adjudication-receipts.md
check each threshold it justifies is pinned in gates.json with provenance, then archive
experiments/612-uk-carrier-payload-receipt.md
the "should be a test" case — confirm compare_uk_h5_payload.py covers it, then delete
CONTRACT_FINDINGS.md
#462 findings; the fix is merged and gated. Delete
Sweep the same pattern outside the UK lane (COVERAGE_PROGRESS.md, PROGRESS_BUILD*.md, _698-*.md, _*LANE-NOTES.md, FINAL_REPORT.md) as a separate cross-lane PR.
B: every imputation stage gated on its own receipts, including the CGT donor stage, whose support bounds are either wired at stage time or re-derived post-redraw rather than left declared-but-dead; the family-coverage guard passes against the spine plan unchanged; uk_input_mass_parity references the spine's own certified line and uk_qrf_tail_concentration is armed from L3, each with run provenance.
C: the national build runs from the microcosm spine with no policyengine-uk-data artifact as an input; the June path and the CGT injection are retired; the candidate is microcosm_uk_2024; no committed Markdown asserts an invariant not also enforced by a test or gate.
The whole-spine parity instrument still runs green against the frozen reference, as the historical record of what the swap changed.
Carried forward, not blocking
owned_land reviewed exclusion expires 2026-09-20 — calendar-critical, independent of this issue.
would_claim_uc frozen at 0.55 (U8; uk-data#452) — the pre-registered lever if an armed run's UC fit is strained.
Parent: #145 (workstream E of the UK-migration epic #665). Successor to #686. Reconciled 2026-08-25 with the #623 calibration-seam plan after the first armed calibration campaign (9 runs on the
uk-first-calibrated-run-623-686build branch).Scope, and what moved out
This issue owns the spine lane after #747 merges: the defect batch the calibration campaign surfaced (A), the per-step gating regime for the spine build (B), and the W9–W11 retirement plus dead code/markdown cleanup (C).
Moved out to #623 (PR #743): the calibration-only driver. The new production calibration seam (
tools/calibrate_uk_national_dataset.py+calibration_seam.py) is the "national driver becomes calibration-only" step — it takes whichever h5 it is given, materializes measures, solves weights under doctrine, and never modifies a data variable. What stays here is the other half: retiring the legacy June path (tools/build_uk_national_dataset.py, with its SPI re-derivation and imputation stages) once the seam is proven, which is spine-lane cleanup per the 2026-08-24 ruling.Gate: nothing here starts until #747 is reviewed and merged. Every item either modifies the spine build (invalidating #747's licensed evidence) or retires a fallback the comparison depends on.
State at the point of handover
signed_parity— 26 beyond-band divergences all signed, 0 unsigned,--strictcleanA. Spine defect remainder
Two of the five landed in #747 rather than waiting for a rebuild here — the review made the case that fixing the spine before the swap is cheaper than certifying it twice:
12 structural-NaN SPI person columnsfixed in Whole-spine parity for the UK: re-pinned incumbent, comparison instruments, signed register, and the E10 evidence (#686) #747: the SPI stage now writes the adjudicated stage-time zero on the FRS channel, so the artifact loads and the calibration seam's NaN fence can stay fail-loud without refusing every spine artifact. This no longer gates the first production calibration.85+ age-tail disaggregationfixed in Whole-spine parity for the UK: re-pinned incumbent, comparison instruments, signed register, and the E10 evidence (#686) #747 as theage_taildeclarative source stage, with the six ONS band populations committed as a resource whose every cell records the register target id it must agree with, and the campaign'sperson_source_idclone-twin keying preserved.Still open here:
spine-c.build.jsoncarries the full stage census, artifact/resource pins, seeds and rules-engine version — and no reader joins it today. Downstream consumers should inherit provenance from the sidecar rather than re-deriving it; wire the join and pin the h5 to its sidecar.slc.repayments.england_postgraduatehas zero support on the spine; the top-income (£1m+) channel holds 2 records on both datasets. Each needs a measured support channel or an explicit adjudication that the gap is accepted.B. Per-step gating regime for the spine build
The 2026-08-24 ruling: no imputation or spine gating in the calibration path — the seam runs a calibration-scoped battery only. That is only coherent if the spine build polices itself, stage by stage, the way the US builds do (#327 doctrine):
Per-stage imputation-health gates. Each imputation stage gets a gate consuming its own receipts (support bounds, incidence vs donor, conservation) so a regression is caught at the stage that caused it, not at a whole-spine diff. The signed-differences register stays the swap-decision record; the per-step gates are the standing health checks.
Wire the dormant CGT donor support bounds — or decide they cannot be terminal.
cgt_band_donor_support_bounds.jsonis declared incountry_package.jsonand read by nothing; the other four support-bounds resources (was_wealth,lcfs_consumption,etb_vat,etb_services) are each listed in theuk_supportgate'ssupport_bounds_resourcesand inbattery_bindings.py. The CGT one is not an oversight — the resource records why in its ownsemantic_fit_note: its intervals describe donor-stage initial support, and E8'shmrc_cgt_gains_spineTable-3 redraw moves amounts onto a different band surface afterwards, so "terminal support-gate applicability requires reviewer confirmation".That makes this the sharpest argument for per-step gating rather than a stray wiring chore. The other four work as terminal gates because nothing rewrites their columns after they clip to donor; CGT is the one stage where a terminal gate provably cannot express the constraint. Two ways out, and the choice is the adjudication: gate the donor stage at stage time, before the redraw, on the bounds as they stand; or derive post-redraw bounds from the Table-3 surface and gate terminally like the others. Either way the resource stops being declared-but-dead. (This also closes the third of the three sign-offs left open on Port the CGT structure, salary sacrifice, and student loan stages onto the UK spine at FRS 2024-25 #740.)
Re-home the family-coverage contract.
uk_release_family_build_stagesrequires 12required_at_buildfamilies; the spine driver builds 10, the June driver 2, so neither driver alone can pass — the W9 seam. Fix by pointing the guard at the spine plan that actually builds the families (the spine'shmrc_spi_income_spine/hmrc_cgt_gains_spinesupersede the June driver's two). Never by relaxing the gate, and never by sidecar plumbing —family_coverage status='required_at_build'is an executable contract, not documentation.Arm the thresholds against the spine's own line (US Export input-mass-parity gate references the raw base, flagging correct calibration gains on PUF-imputed columns #327: a parity gate compares a release to its own previous certified line, not a foreign artifact): flip
uk_input_mass_parity's reference to the spine's first certified release; armuk_qrf_tail_concentrationfrom the Assemble the microcosm-built UK spine: whole-spine parity and swap acceptance #686 L3 baselines (47 QRF tail grids, 12 household); RC battery +macro_realismon the swap build. Every armed threshold records which run measured it.Release-cut certification. The microcosm-data release-verification contract expects the full 21-entry battery on a published artifact — that expectation maps to the monolith. Combining spine-side per-step gating + the seam's calibration-scoped battery into a publishable certification is contract evolution; until it lands, seam-built records say
shippable: falsewith the reason.C. W9–W11 retirement, dead code, markdowns
tools/build_uk_national_dataset.py's SPI re-derivation,replace_uk_spi_support_tables's zero-weight-channel refusal, and the CGT redraw all encode the June convention the spine has superseded.UK_CANDIDATE_DATASET_NAME→microcosm_uk_2024(uk_runtime/terminal_gates.py:72, currentlypopulace_uk_2023). Ruling 2026-08-25: the candidate carries the microcosm name, not the legacy populace one — superseding the earlierpopulace_uk_2024note; the repo itself has moved toPolicyEngine/microcosm, and the parity-instrument tests already usemicrocosm_uk_2024.h5as the canonical candidate filename. Sweep the straypopulace_uk_2024*fixture names intest_uk_rowwise_build_driver.pyin the same pass. CheckUK_REFERENCE_DATASET_NAMEalongside: once the spine is the build, "reference" means the previous certified microcosm line, not the enhanced-FRS artifact.microcosm-data/contract.py:219anduk_runtime/weighted_integrity.py:241once nothing reads it as an input. The parity reference itself stays — it is the historical comparison instrument and keeping it is how the swap remains auditable.UKPolicyEngineAdapter(zero production callers — the harness rebuilt its capability outboard); the assessment harnesstools/calibrate_uk_national_spine.pyretires once the seam driver covers it..mdis a claim nobody re-checks; promote to a gate/test, make regenerable, or archive with a date:UK_COVERAGE_PROGRESS.mdrelease_input_coverage_manifest.json. Delete.experiments/686-uk-spine-comparison-ledger.mdexperiments/686-uk-spine-swap-receipts.mdexperiments/630-uk-gate-adjudication-receipts.mdgates.jsonwith provenance, then archiveexperiments/612-uk-carrier-payload-receipt.mdcompare_uk_h5_payload.pycovers it, then deleteCONTRACT_FINDINGS.mdSweep the same pattern outside the UK lane (
COVERAGE_PROGRESS.md,PROGRESS_BUILD*.md,_698-*.md,_*LANE-NOTES.md,FINAL_REPORT.md) as a separate cross-lane PR.D. Acceptance
uk_input_mass_parityreferences the spine's own certified line anduk_qrf_tail_concentrationis armed from L3, each with run provenance.policyengine-uk-dataartifact as an input; the June path and the CGT injection are retired; the candidate ismicrocosm_uk_2024; no committed Markdown asserts an invariant not also enforced by a test or gate.Carried forward, not blocking
owned_landreviewed exclusion expires 2026-09-20 — calendar-critical, independent of this issue.would_claim_ucfrozen at 0.55 (U8; uk-data#452) — the pre-registered lever if an armed run's UC fit is strained.