Two-country catalog: UK seven-source harvest, campaign PE results, baselines registry (#13), descriptive register (#9)

[MaxGhenis]

Completes the two-country catalog: the seven-source UK harvest ingested with the Ledger routing rule, the campaign's UK compute batch joined as PE results, the baselines registry (#13), and the descriptive-register gate (#9). Reconciled on top of this week's main (PRs #24–#30): main's ingest_campaign stays canonical for the day-1 US families; this branch adds its UK sibling and the schema layer both now share.

Report

Claims in DB: 74,237 across 66 sources (regenerated via the documented order: urban → platform → solo → diagnoses → harvest → reform_validation → uk → campaign → campaign_uk). The eight UK sources this PR adds:

source	claims		source	claims
obr	25,425		ukmod	1,392
dwp	1,952		hmrc	987
hmt	1,368		jrf	300
ifs	268		resolution_foundation	70

UK full accounting — 33,943 staged rows = 31,762 ingested + 2,090 routed to Ledger staging (data/ledger/uk_admin_outturns.jsonl: HMRC outturn/provisional 1,855 · OBR EFO outturn columns 133 · DWP BECL outturns 102, deterministic pre-agreed fact ids for the populace/Ledger lane) + 89 deliberate drops tallied in lane notes (88 JRF MIS budget cells, 1 RF price-index row) + 2 JRF rounded/unrounded twins merged with rounding-consistency assertions.

Campaign UK results joined — 65 PE results via scorecard_db/ingest_campaign_uk.py, every one recording the baseline actually executed (#13): uk_reckoner 23 · uk_free_joins 23 · uk_obr_measures 10 · uk_uprating 6 · two_child_pentagon 3. Plus 11 two-child-limit pe_exhibits (calibrated + entitlements-basis worlds + abs-line continuity pair) against the registered pre_ab2025 baseline, and 8 methodological_difference diagnoses on the CPSP level rows (populace#593 cited). Two honest uprating skips (LCWRA new/existing split, data-driven LHA) land not_computed with the tree-check's own notes. DB totals after everything: 9,438 pe_results, 535 exhibits, 19 diagnoses, 13 registered baselines, zero unregistered.

Tests: 200 green (uv run --with pytest pytest tests/ -q, the CI gate): +12 schema (registry, view guard, diagnose gate, migrations), +42 UK adapters (FY normalization, ledger routing, full accounting, verified consumed sets), +17 UK campaign joins, alongside main's new reform-validation/campaign/export/moments suites. ruff format --check clean.

Schema (#13, #9)

• baselines registry table + baseline_key on external_scores / pe_results / pe_exhibits, carried through the shared SQL constants/preparers so main's single-transaction ingests record it too; curated in scorecard_db/baselines.py (13 worlds — the deliberate-registration gate caught reform-validation's pre_obbba_current_law during the merge, exactly as designed). Claim-id hashing unchanged.
• comparisons view exposes claim/PE baseline labels + pe_status_effective: comparable across different baselines can never render as plain agreement (observable on TPC's Senate-Title-VII claims).
• DiagnosisClass.methodological_difference (strictly descriptive); pe_gap/external_issue now require a citable action_link — db.diagnose raises otherwise.
• UK vocabulary: gbp unit family, exchequer_impact (deliberately not revenue_change — Tax/Spend heads mix), income-distribution and taxpayer/percentile/average-tax families; poverty_count + persistent_poverty_rate join the permanent-holdout set.

UK ingest decisions worth review

• period = FY start year everywhere, re-derived from verbatim fy labels (fixes DWP FYE end-year keying and the 1999/00→1900 staging defect).
• consumed_as_target only on literal pe-uk-data consumption, read this session: obr.py receipts + Table 4.9 welfare lines (Nov-2025-vintage caveat named), BECL expenditure for the five exact-line benefits, DWP UC admin + PC take-up seed; HMRC SPI-family projections are seed_source via the shared SPI 2023-24 base; HB take-up stays held out as the live comparator for policyengine-uk#1813.
• Reconstructed identity axes the staging dropped (workbook order preserved): OBR 4.9 welfare-cap sections, 4.11 parent lines, duplicate-titled measure component runs, the Mar-2026/Nov-2025 EFO vintage split, IFS recipient-group columns.
• The campaign's DWP free-join legs anchor FY2025-26 cells (the runner selected staged end-year ints); results attach to the claims the values identify, with the vintage caveat annotated rather than hidden.
• The campaign's staged UK descriptor files are deliberately superseded rather than attached (strict-subset coverage; under-determined reckoner descriptors) — except the two-child pentagon rows, which land translated. Rationale in the module docstring.

Notes / follow-ups

• data/scorecard.db is 94 MB (was 40 MB) — under GitHub's 100 MB hard limit but the next population will cross it; publication-dict normalization or moving the binary out of git needs a decision.
• Cross-source canonical reform slugs (the two-child pentagon legs remain per-source worlds) deferred to the adjudication phase.

🤖 Generated with Claude Code

[MaxGhenis]

Closing per the two-model adjudication (fable + sol independent cold-reads, both HIGH confidence): the uk/* adapter stack was chosen as the UK foundation, and this PR's durable contributions were extracted rather than merged wholesale. Credit ledger — what this PR contributed to what shipped:

• Baselines registry (Baseline as a first-class attribute of every score #13) and the cross-baseline view guard: this PR built them first. Extracted, hardened through a four-round gate, and merged as Baselines registry + descriptive-register gate (issues #13, #9) #71 (52 registered worlds now 53; the guard is end-to-end through the exporter and app).
• Descriptive register (App design: public scoreboard + mission control, four views #9): the diagnose-gate (pe_gap/external_issue require a citable action link) came from here, also via Baselines registry + descriptive-register gate (issues #13, #9) #71 — it caught a real CPSP erratum on the way in.
• Seven-source UK harvest breadth: the 2026-08-02 snapshot is archived intact under sources/harvest-uk-2026-08-02 (Archive the 8/2 UK harvest snapshot + campaign UK runs (source-only) #70) with its NOTES preserved; the five primary-source adapter lanes (DWP take-up adapter (mode 1): FYE 2024 external side #43–UK externals → DB ingest path (16,924 claims, five sources) #48, merged) re-built the ingested slice from vendored primary sources, and the remaining breadth (IFS, RF, JRF, HMT, OBR costings) stays staged there for the long-tail lanes, pending the DB-storage decision.
• Campaign-UK results: superseded by the staged-descriptor doctrine — the adjudication found this PR's ingester recomputed deltas and superseded the campaign's own staged rows, so the replacement is a deterministic producer that resolves the archived campaign staging against the ingested claims via the strict claim-id match form (14 reckoner rows attachable; the free-joins/OBR-measures/RF families blocked on their source lanes, reasons machine-readable in the producer).

Adjudication records: the two independent verdicts and the sequenced plan are in the #48 thread; #48 merged today as the foundation with this PR's schema layers already under it.

🤖 Generated with Claude Code