OBR costings divergence decomposition: axes registry + identity-checked pipeline (#59)

[vahid-ahmadi]

The #30 pattern applied to the mode-2 UK lane: every #56 comparison row decomposes into signed GBP components summing exactly to gap = PE − OBR, with a residual, under two honesty rules enforced by the pipeline and tests — a sized component must carry per-FY values with verbatim fetched-primary-source provenance, and any unsized component forces decomposition_status: partial with the remainder labelled residual_plus_unsized, never explained.

What's sized with real published numbers (fetched, quoted in the registry)

• Employer NICs package — missing Employment Allowance construction: +£4,581m (2026-27) / +£4,631m (2027-28) and static-vs-post-behavioural: +£723m / +£703m, from the OBR's 8 May 2025 supplementary release Static costing of changes to Employer NICs (Tables 1.3, 1.4, 1.5; https://obr.uk/docs/dlm_uploads/Cosings-supp-NICs-AB24-1-1.pdf). The sized components expose the real finding: the residual −£12.67bn is the like-for-like static gap between PE's employer-NICs wage-incidence machinery and HMRC's PTM static of the rate+threshold elements — a policyengine-uk development item, per the registry's residual recipe.
• HICBC — missing welfare head: +£391m / +£426m (the comparison's own head rows; PE emits zero on that head).
• The SB2024 Policy Costings document's final Exchequer rows and verbatim behavioural-response descriptions (employee NICs cut, HICBC) are quoted as provenance for the unsized behavioural axes, with sizing recipes (the OBR supplementary-release route has precedent, and the reckoner lane HMRC ready reckoner (mode 2): PE-UK counterparts to the direct effects of illustrative tax changes #60 is the other instrument).

Files

• data/uk/obr_divergence_axes.json — 6-measure registry (JSON, not YAML: CI installs pytest only). Axes: behavioural_adjustment, baseline_vintage, cy_proxies_fy, head_scope, construction_scope, residual.
• pipeline/decompose_uk_obr_divergence.py — engine-free csv/json arithmetic; emits results/uk/obr_divergence/DECOMPOSITION.{json,md}; a measure whose registered row_kind is absent errors rather than decomposing a different scope.
• tests/fixtures/obr_costings_comparison_20260817.csv — verbatim copy of origin/obr-costings-mode2:results/uk/obr_costings/COMPARISON.csv (provenance in the test module docstring) so the pipeline is executable before OBR policy costings (mode 2): registry, offline certified compute, staging, descriptive comparison #56 merges.
• tests/test_obr_divergence_decomposition.py — 10 tests: registry contract (sized⇒provenance, unsized⇒recipe), the exact identity, partial-never-explained, the pinned employer-NICs and HICBC sizings, missing-row refusal.

Suite: 173 passed / 4 skipped; ruff clean.

Dependency: consumes #56's COMPARISON.csv by path only — no shared commits; when #56 merges, running the pipeline against the real artifact replaces the fixture path. 12/12 decompositions are partial by design until the unsized axes get their runs.

Builds #59.

🤖 Generated with Claude Code

[DTrim99]

Review — OBR costings divergence decomposition

The mechanical scaffolding (schema, "never explained while unsized" gate, JSON hygiene) is sound, but I think the decomposition has a substantive correctness problem that the current test locks in rather than catches. Requesting changes.

The identity "check" is a tautology

decompose() (decompose_uk_obr_divergence.py:293) defines residual = gap − sum(sized), so sized + residual == gap can never fail — and test_arithmetic_identity only re-checks that. The honest test (that independently-sourced sized components don't exceed or oppose the gap) is absent.

Working the employer-NICs 2026-27 row by hand: gap = PE − OBR = 16,246,687,341 − 23,610,420,262 = −£7.364bn. Σsized = +£4,581m + £723m = +£5.30bn, so residual = −£12.67bn — larger in magnitude than the gap and opposite in contribution. The "sized" terms move away from OBR, which is the opposite of a decomposition that explains divergence.

Sign / attribution of the sized components

Both employer-NICs sized components (data/uk/obr_divergence_axes.json:28-36) are attached to mapped_head_total (IT + NICs) with the convention "+ = PE above OBR" — but on that row PE is £7.36bn below OBR. The +£4,581m "missing Employment Allowance construction" claim (PE lacks EA ⇒ PE higher) is contradicted by the data: the NICs head alone has PE only +£0.83bn above OBR, and the total is dominated by the −£8.19bn IT head. So the component appears sized against the wrong row, and its direction argument doesn't hold where it's applied. test_employer_nics_sized_components hard-codes these as correct.

Should

• The behavioural component (+£723m/+£703m, axes.json:35-36) is derived (static Table 1.5 minus summed PMD rows) and the provenance itself admits it "bundles the behavioural response with any static-vs-PMD head-allocation mismatch." That breaks the PR's own honesty rule ("sized ⇒ verbatim fetched primary-source value") — it should be unsized with a recipe, or a new derived status. load_registry only checks presence of values_gbp + provenance, never that the number is quoted.

To be clear on what's fine: the "never mark explained while an unsized component exists" rule is enforced in code and tested — that part is good. The gap is that presence is enforced but truth (right row, right sign, quoted-not-derived) isn't.

Reviewed with Claude Code assistance.

[vahid-ahmadi]

All three points addressed in ff38e0e, including the substantive one. Point by point:

The tautology. Agreed — residual = gap − Σ(sized) can never fail and the old test_arithmetic_identity only re-checked its own definition. It's now bookkeeping, labeled as such, and the record carries diagnostics that can actually fail: every valued component is classified explains_gap / masks_gap against the gap's sign; any masking component sets divergence_understated: true with like_for_like_gap_gbp (the gap with the masking axes stripped); residual_exceeds_gap flags |residual| > |gap|; and explained_share is emitted only for complete decompositions with no masking components. Synthetic tests pin each: a component opposing the gap must classify as masking with no explained claim (this test fails on the old semantics), and your hand-worked employer-NICs row is a committed regression — gap −7.364bn, Σvalued +5.304bn both masking, residual −12.668bn, residual_exceeds_gap and divergence_understated both true, no explained_share.

Sign / attribution. I re-checked the orientation chain and the two components' signs are individually defensible (EA-absence contributes +4.58bn to PE−OBR because the missing elements are revenue-reducing in OBR's leg; static−post-behavioural is +723m by Table 1.5 arithmetic) — but you're right that presenting them as decomposition steps on this row was wrong, and right about the head attribution: the EA effect lands on the NICs head where PE is only +0.83bn above OBR, so ~−3.75bn of offsetting NICs-base factors must coexist, and the total gap is dominated by the IT head (PE −8.51bn vs OBR −0.31bn) — which is PE's employee_incidence = 1 construction, previously not a component at all. The registry now: (a) carries the head-attribution caveat verbatim on the EA component, (b) adds employee_incidence_and_base as the named dominant component (unsized, paired-run recipe: re-run with employee_incidence = 0, difference per head; the same runs isolate the NICs-base offset), and (c) the artifact's honest statement for this measure is that the observed −7.36bn understates a ~−12.67bn like-for-like disagreement — a masking record, not an explanation.

Derived vs quoted. The behavioural component is re-statused derived: load_registry now requires a mandatory derivation formula (which travels into the artifact) and rejects any sized component carrying one — sized means verbatim-quoted only, enforced and tested both ways.

Tests 10 → 17; full suite 180 passed / 4 skipped; ruff clean.

[DTrim99]

Re-review — request-changes resolved ✅

This fully addresses my earlier review — nicely done, and thank you for the rework. Verified against the live pipeline (17 tests pass; 12 records, all partial, 0 with explained_share, 4 masking).

• Identity tautology — the residual = gap − Σ(valued) identity is now explicitly labelled bookkeeping ("NOT evidence"), and real, failable checks replace it: direction, residual_exceeds_gap, divergence_understated, and explained_share emitted only when complete + unmasked + gap≠0 (never on these rows).
• Sign/attribution — the EA (+£4.581bn) and behavioural (+£0.723bn) components now classify as masks_gap against the −£7.364bn gap (both opposite-signed), with divergence_understated: true and like_for_like_gap = −£12.668bn. I reproduced the arithmetic. The wrong-row concern is now surfaced via a head-attribution caveat + an unsized employee_incidence_and_base component naming the −£8.5bn IT-head driver with a paired-run recipe — pinned by test_employer_nics_row_matches_the_hand_computation.
• Derived-not-quoted — new status: derived with a mandatory derivation, and load_registry enforces derived⇒derivation and sized⇒no-derivation, so a computed number can't masquerade as quoted.

Remaining bits are honestly disclosed and non-blocking (the EA head-attribution still awaits the paired run to confirm; a render-only unicode console quirk). Happy to see this move forward — clearing my change request.

[DTrim99]

Re-reviewed — my earlier change request is fully resolved: the identity is de-tautologized with failable checks, the employer-NICs components are reclassified masks_gap (I reproduced the arithmetic), and the behavioural term is status: derived under an enforced sized-vs-derived contract. Approving — thanks for the thorough rework.

[MaxGhenis]

Gate round 1 — the arithmetic closes; the decomposition doesn't yet

Clean: exact arithmetic closure on all 12 fixture rows, 13 focused tests, formatting.

1. Blocking — the focal divergences remain undecomposed. Every PA/HRT-freeze and additional-rate component is unsized; the residuals equal the entire gaps (£13.7bn/£15.0bn and £0.90bn/£0.96bn), and no record emits explained_share. FY2026-27's CY component is already computable from the fixture's adjacent runs under the PR's own 3:1 recipe (+£1.443bn PA/HRT, +£23.5m additional-rate) — recipes must be executed, not asserted.

2. Blocking — "identity-checked" is bookkeeping, not verification. Residual is defined as gap − sum(components), so closure is guaranteed even for unsupported components. Specifics: employer-NICs subtracts a full-measure static total from a mapped-head post-behavioural subtotal (a joint scope difference, not a behavioural contribution); Employment Allowance sums two OBR cells but is labeled verbatim sized (it's derived); HICBC is a sign-adjusted head difference that also overlaps the unsized take-up axis; residual exists both as an axis and a computed quantity; the synthetic test permits explained_share = 1.5.

3. Blocking — axis definitions contradict the engines. The registry claims OBR has "no comparable pass-through," but the Oct-2024 EFO assumes 60% (2025-26) then 76% employer-NICs pass-through with indirect effects reported separately — a PE 1→0 incidence pair sizes PE's whole wage channel, not a PE-vs-OBR wedge. The PA/HRT recipe substitutes March-2026 earnings determinants PE 2.89.2 already uses for uprating (redundant, and conflates data uprating with threshold indexation). The claimed announcement baseline isn't registered as a world, and the CY recipes ignore the FY-end-year identity convention (3:1 is an equal-accrual approximation, not an engine fact).

4. Blocking — doesn't stand alone or persist. Default CLI requires OBR policy costings (mode 2): registry, offline certified compute, staging, descriptive comparison #56's absent COMPARISON.csv (clean invocation FileNotFoundErrors); tests use a frozen fixture, so OBR policy costings (mode 2): registry, offline certified compute, staging, descriptive comparison #56's baseline fixes won't propagate; nothing registers in build_db.py, so outputs vanish on CI rebuild.

5. High — disposition and register. Computed paired-run differences belong in pe_exhibits with full derived-pair exhibit_meta + baseline/engine/bundle/run ids; diagnoses need a canonical OBR policy costings (mode 2): registry, offline certified compute, staging, descriptive comparison #56 claim anchor with class methodological_difference (descriptive, no action link needed — unlike pe_gap). "Dominant driver," "overstates," "over-representing" must stay neutral hypotheses until measured.

The shape to aim for: each axis = an executed paired run (issue #78's job), sized contributions in exhibits, and explained_share computed only from executed components — that's the metric this lane exists to move.

🤖 Generated with Claude Code

[vahid-ahmadi]

All four addressed in a7d901a.

1. Executed, not asserted. You were right that the FY2026-27 CY component was already computable from the fixture's own runs. Recipes are now EXECUTED: a computed component names an executor the pipeline runs against the comparison rows on every run and may not carry a hard-coded value at all. The adjacent-CY 3:1 interpolation sizes both focal measures — +£1.443bn on the PA/HRT freezes and +£23.5m on the additional-rate reduction, matching your hand figures — and both residuals are now smaller than their gaps. FY2027-28 needs a CY2028 run the fixture does not have, so it stays unsized and names the missing run rather than extrapolating.

2. Bookkeeping → something that can fail. residual is no longer an axis; it was both a registry axis and the computed quantity, so a component could be filed under the name of the thing it explained. The employer-NICs term becomes static_vs_post_behavioural_joint on a new joint_behavioural_and_scope axis — it takes a full-measure static total from a mapped-head post-behavioural subtotal, so it carries scope and basis together and cannot be attributed to behaviour. Employment Allowance becomes derived, since it sums Tables 1.3 and 1.4 and the source prints neither total. HICBC's welfare_head_missing declares that it overlaps its own take-up axis; overlapping valued components are recorded and withhold explained_share. And a share outside [0, 1] is withheld with a stated reason — the synthetic test asserting 1.5 was asserting a residual of the opposite sign wearing an explanation's name.

3. Axis definitions. Corrected. The recipe no longer claims OBR has no comparable pass-through: the October 2024 EFO assumes 60% rising to 76% with indirect effects reported separately, so a PE incidence 1→0 pair sizes PE's whole wage channel and the wedge needs OBR's separately-reported leg too. The PA/HRT recipe no longer proposes substituting earnings determinants 2.89.2 already uses, and separates data uprating from threshold indexation; the counterfactual it actually needs is now a registered world (obr_announcement_baseline_efo_march_2026) — you were right that it was claimed as a baseline while registered nowhere. Every CY derivation states the equal-accrual assumption and the FY-END-year identity convention.

4. Stands alone and persists. A clean invocation now falls back to the committed fixture and SAYS SO, in the output and in a PROVENANCE.json, and compares the fixture against OBR policy costings (mode 2): registry, offline certified compute, staging, descriptive comparison #56's live file when both exist so a frozen snapshot cannot silently miss later baseline fixes. scorecard_db/ingest_obr_divergence.py registers the lane in build_db, and writes it as "not run" when no decomposition exists rather than writing nothing.

Suite 277 passed, two builds agree on content_hash, no-drift clean, ruff format clean.