Add council-tax nation-total and CTR-cost source packages (Scotland, Wales, England)

[MaxGhenis]

Summary

Nine UK source packages carrying the nation-level council tax billed/collectable totals and council tax reduction (CTR) costs that the policyengine-uk-data#448 council-tax diagnosis compared the model against. Chronicle's seven existing council-tax packages are levels, Band D rates and band stocks; none carried a billed or collectable total or a CTR cost. Every fact is a publisher cell (stored value × declared value_scale) with exact row/column header guards read from the pinned bytes.

package	publisher table	facts
scotgov-council-tax-collection-2025-26	Council Tax Collection Statistics 2025-26, publication tables, Figure 1 (CTRR): net amount billed, amount received, uncollected, percentage received, 1999-00 to 2025-26, Scotland	108
scotgov-council-tax-collection-2024-25	the same table, 2024-25 vintage (1999-00 to 2024-25, as at 31 March 2025)	104
scotgov-slgfs-council-tax-2024-25	SLGFS 2024-25: Table 2.11 (CTR funding from SG, final total reduction in liability), Table 2.8 (council tax income after CTR), Table 2.6 (the same income 2020-21 to 2023-24), Chart 2.7 (potential yield by band: billed estimate, CTR, single person discount, exemptions, other discounts, gross), Table 2.10 (average bill before/after CTR, 2020-21 to 2024-25)	71
welshgov-council-tax-collection-2025-26	Council tax collection rates in Wales 2025-26, Table 2, Total Wales: arrears brought forward, prior-year debits/credits, arrears collected and written off, in-year net collectable debit (net of CTRS), amount collected, written off	7
welshgov-council-tax-collection-2024-25	the same table, 2024-25	7
welshgov-ctrs-annual-report-2025-26	Council Tax Reduction Scheme annual report 2025-26 (PDF), Table 2 total value of CTRS awards in Wales, 2024-25 and 2025-26	2
welshgov-ctrs-annual-report-2024-25	the same table in the 2024-25 report, 2023-24 and 2024-25	2
mhclg-council-tax-levels-england-summary-2025-26	Council Tax levels set by local authorities in England 2025-26, Tables 1-9 (re-issued 22 April 2026), Table 1: requirement excl./incl. parish precepts, parish precepts, taxbase, average Band D, 2021-22 to 2025-26	30
mhclg-council-tax-collection-england-2025-26	Collection rates for council tax and non-domestic rates in England 2025-26 (re-issued 31 July 2026), Table 5: council tax net collectable debit and amount collected in year, 2021-22 to 2025-26	10

Headline values (stored cell × scale; the diagnosis quoted the rounded display values of the same cells):

nation	measure	fact
Scotland 2025-26	net amount billed after CTR, excl. water and sewerage / received by 31 March 2026	£3,388,883,606 / £3,226,613,159 (95.2%)
Scotland 2024-25 (in-year vintage)	net amount billed / received by 31 March 2025	£3,076,769,838 / £2,938,384,739
Scotland 2024-25	CTR final total reduction in liability (SLGFS Table 2.11) / CTR in the potential-yield breakdown (Chart 2.7) / council tax income after CTR (Table 2.8)	£389,010,000 / £390,302,660 / £2,996,849,000
Wales 2025-26 / 2024-25	net collectable debit excl. CTRS / collected in year	£2,523,751,000 / £2,403,994,830 ; £2,331,585,000 / £2,228,016,000
Wales 2023-24 / 2024-25 / 2025-26	total value of CTRS awards	£301,121,000 / £322,938,000 / £347,393,000
England 2025-26	council tax requirement incl. ASC and parish precepts / taxbase / average Band D	£44,118,104,184 / 19,348,236 Band D equivalents / £2,280.21
England 2025-26	net collectable debit / amount collected in year (MHCLG collection rates)	£44,981,384,000 / £43,002,543,000

Conventions

• Publisher provenance: every artifact is pinned byte-for-byte under db/data/… with manifest.yaml (canonical publisher source_url, sha256, storage.r2 location). The nine raw files are uploaded to the private ledger-raw bucket at those keys (chronicle publish-raw, 2026-08-23; three spot-checked by re-download and sha256). gov.wales 403s a plain fetch; the workbooks and PDFs were retrieved with browser headers.
• Untransformed values: cells are carried as stored, not as displayed (the Scottish CTRR cells are full-precision floats behind #,##0 formats; SLGFS stores £351,000 thousand as 350999.99999999977). value_scale is 1000 for £ thousand tables and 1,000,000 for £ million tables; the Scottish percentage received is the publisher's fraction (unit: share).
• Periods and geography: fiscal_year with the opening year (2025-26 → 2025), one record set per year because a period is not a measure; geography is the nation each table covers (S92000003 / W92000004 / E92000001); amounts carry government/billing_authority (the Welsh levels precedent) and per-dwelling averages and the taxbase carry dwelling/chargeable_dwelling.
• Exclusion rule, stated in every header comment: within the selected table, only cells the publisher itself labels as arithmetic over other cells are left out (Welsh columns 5=(1+2-3-4), 8=(7/6), 10=(6-7-9), 11=(10+5), 12=(11-1); MHCLG/SLGFS "percentage change" rows and columns); out-of-scope rows and tables (per-authority rows, non-domestic rates, caseload and percentage tables) are named per package.
• PDF source: the Welsh CTRS cost is printed only in the annual report PDF, which is the artifact and is parsed with the repo's pdf_text_numbers parser (LIHEAP/ISC/PIP precedent), guarded by page, line, number text and the table-title line.
• Vintages: the two Scottish CTRR workbooks are two complete publisher tables; 1999-00 to 2024-25 therefore appear in both with restated values, which the bundle reports as duplicate_semantic_fact_key warnings (the designed "consumers select by source_table" path; the golden count moves from 12 to 117: 104 from the Scottish vintages and one from the CTRS 2024-25 value printed in both reports).
• Registration: alias map (chronicle/source_package.py), curated UK bundle (chronicle/bundle.py), harness-doc command block, a new "Beyond parity" section in docs/pe-uk-source-checklist.md, focused tests in tests/test_chronicle_council_tax_nation_totals.py, and the merged-bundle goldens.

Not ported (documented, not invented)

• England LCTS cost in £: MHCLG publishes LCTS as a taxbase deduction, not an amount (not found in the levels or taxbase releases checked).
• Scottish per-authority rows (SLGFS 2.8/2.11, CTRR worksheets), Welsh per-authority rows and the Welsh percentage-rate tables: wave-3 local-geography scope or rates, all inside the pinned artifacts.
• MHCLG collection Tables 1 (receipts irrespective of year) and 2 (rates by class); the 2024-25 collection release, whose overlapping values equal the 2025-26 release's (it adds only 2020-21).
• One citation correction for _uk-defects-452-448/measure/COMPARATORS.md: the "CTR ≈ £389m (2024-25)" row cites the gov.scot news release, which carries no CTR amount (its only "389" is "£3.389 billion"); the figure is SLGFS 2024-25 Table 2.11, Scotland row, £389,010 thousand — now a Chronicle fact.

Activation in Microcosm (decided 2026-08-23 with a fable + sol cross-family review; follow-up PRs open)

• PolicyEngine/microcosm#754 (draft): scoped Scotland pin for the nine scotgov.council_tax_stock.* holds, regenerated against the byte-identical feed vintage (chronicle 1cab809, sha 226358e7…); membership moves by exactly those nine targets (388 → 397 active).
• PolicyEngine/chronicle#192 (draft, stacked on this PR): OBR Table 4.1 rows 15/17/18, so the UK council-tax binding can be redesigned on a household-comparable basis before the four obr.council_tax_* / obr.domestic_rates holds are activated (see below).
• The four OBR nation holds stay held until that redesign: the active obr.council_tax UK total (row 19, £50.925bn) is the three nation rows (£49.569bn) plus NI domestic rates (£0.494bn) plus an accruals adjustment (£0.862bn) with no household counterpart, so activating the decomposition beside it would be jointly infeasible. CTR under-simulation (Scotland £249m modelled vs £389m; Wales ~£10m vs £323m) is the lever to fix first.

Original analysis, kept for the record:

These packages do not by themselves activate anything. At microcosm 2aa96795 the held ids obr.council_tax_{england,scotland,wales} and obr.domestic_rates select source_name: obr and Chronicle already emits their facts (obr-efo-expenditure-march-2026, FY2024–FY2030); they are held because tools/generate_uk_target_references.py::_geography_id_for_target pins them to the nation (E92000001 / S92000003 / W92000004 / N92000002) while the OBR rows are stamped K02000001, and geography_id is part of the merged selector (target_reference_authoring.py:248) matched against fact.geography.id (ledger_targets.py:1422). The same pin logic sends scotgov.council_tax_stock.* to K02000001 ("scotgov" does not contain "scotland") while the CTAXBASE facts carry S92000003. The reader workflow reproduced every held status with the real code and computed the counterfactuals (pin K02000001 or no pin → active, resolving to the FY2025 source_projection values). Paths, each a microcosm- or chronicle-side change:

1. microcosm: correct the pin derivation or add _selector_pins geography overrides for those ids (activates on the existing feed, resolving to the FY2025 source_projection OBR values under allow_source_projection); or
2. microcosm: re-point the obr.council_tax_* selectors at these packages (source_name: scotgov|welshgov|mhclg, the concepts above), noting that a 2024 target period would see both Scottish vintages (multi_fact unless source_table is pinned) and that the OBR basis (accruals receipts) differs from the billed/collectable and requirement bases; or
3. chronicle: restamp the OBR nation rows with their nation geography (changes those facts' keys).

Chronicle governance

• Approved Chronicle agent role: ledger-source-ingestor (the chronicle/source_package.py and chronicle/bundle.py edits are the alias/bundle registration the drift guards require, as in UK source packages, wave 3: back the uk_local_geography profile (constituency + local authority) #134).
• Deterministic checks run: validate-package and build-suite for all nine packages (agent acceptance valid, 341 facts, R2 pointers and source-cell lineage on every fact); uv run ruff check chronicle policyengine_chronicle db scripts tests; uv run pytest -q (merged-bundle goldens regenerated).
• LLM judge verdicts:
  • ledger-source-fidelity: two adversarial workflow rounds re-read every built fact from the raw bytes round 1: seven per-package refuters re-derived all 333 facts then in scope from the raw bytes (independent openpyxl/XML/ODS parses) and every header and guard string — zero value mismatches; their documentation findings (copied cell examples, an overstated Welsh Table 1 reason, stale date wording) are fixed in this revision; round 2: the two CTRS PDF packages verified CLEAN (151 checks, including an independent pypdf extraction and a parser re-run), the revised SLGFS package re-verified 71/71 values, and a docs refuter caught the Welsh CTRS omission and a wrong issue link, both fixed.
  • ledger-target-profile: n/a (no profiles touched).
  • ledger-contract: n/a (no schema change).
  • ledger-boundary: PASS — publisher cells only; no reconciliation, aging, imputation or activation decisions (the activation note above is a decision for microcosm).

Tests

• tests/test_chronicle_council_tax_nation_totals.py: every headline oracle above pinned to the stored cell, fact counts, geography/entity/period/unit shape per package, the Chart 2.7 gross-row identity, the income-after-CTR series across Tables 2.6/2.8, and the two-vintage restatement property.
• tests/test_chronicle_bundle.py: merged-bundle goldens updated fact_count 155,689 → 156,030, source_package_count 134 → 143, semantic_duplicate_key_count 12 → 117, by_source mhclg 2,672 → 2,712 / scotgov 2,504 → 2,787 / welshgov 198 → 216, by_source_table 129 → 138, plus per-table pins for the nine new tables.
• tests/test_source_package_alias_drift.py: UK bundle/alias parity.

Draft — holding for Max's go.

🤖 Generated with Claude Code