OBR: two-sided overview, and stop three model caveats shipping silent - #182
Merged
Conversation
… microsimulation Step one of the redesign. The nav goes from five items to four — Home · Models · Forecasts · Use — and the two entry pages are restructured so the architecture is legible: pe-microsim is the core, the macro models extend it, and the bridges between them are named. **The merge moves no URLs.** All thirteen /economy pages change section without changing address: `site_nav.section()` maps the economy root to "forecasts" and both country hubs hang off /forecasts. Moving them under /forecasts/ was rejected on a hard constraint rather than on cost — /economy and /economy/us carry ~210 inbound links including ~200 generated release notes, and redirecting would leave those pages linking to a redirect source, which the suite forbids. vercel.json and sitemap.xml are byte-identical. The section is fifteen pages with three jobs and one control bar: an Overview per country (/forecasts, /forecasts/us), a Data hub per country (/economy, /economy/us), and eleven topic pages. The bar reads `Overview · Data · <topics>` under a UK/US scope switch. **Both overviews now open with what you can run** — producible outputs, the model chain in the order it runs, and a command. Four of six UK routes and all four US routes put pe-microsim in the chain: the HHDI_ADDFACTOR bridge into obr-macro, og-uk feeding a second microsim run, and the svar/frb-us/us-hank incidence overlays. Every claim re-derives from the registry on render and RAISES rather than prints if it goes stale — that pe-microsim is the sole household model, the sole two-country model, and that its question types exclude `forecast`. `no_us_forecaster()` still gates the US page, and the two missing US rows quote `cannot_answer` verbatim, because a tab named Forecasts must not imply a US forecaster that does not exist. **The home page no longer leads with a forecast fan** from the one model whose own validation page reports no skill against a naive benchmark. It leads with the pipeline the platform actually is — Reform → pe-microsim → obr-macro → Scored result — then runs the core, the extensions, the joins. Two overlapping grids (four path cards and seven peer verification cards) became one band in which each macro model states what it *adds*. Every caveat from the retired cards survives with its evidence-class tag. **models/index.html loses a duplicate catalog** — it carried a six-item chooser and a separate seven-card grid of the same models — and gains a "Combine" section naming each bridge function. define-uk gets the chooser entry it never had. 2601 → 2374 words. Two numbers were deliberately dropped rather than moved: og+microsim has no committed artifact (only synthetic fixtures), and the duplicate £7.38bn on the obr-macro card has no scored run behind it — it survives once, in #score, beside the caveat that owns it. A stale count is fixed: "five working papers" against six paper slugs, now six. Also: `cli_commands()` only saw options written inline under `@main.command`, so `pe-macro household-impact` looked as though it declared nothing but `--reform`; it now expands the shared decorator, which makes the --country/--people examples verifiable for the first time. And the release-note generator plus 39 generated notes linked to "the Economy page", a tab that no longer exists — relabelled, link only, no observation touched, and five of them pointed at /economy/ with a trailing slash that was itself a redirect source. A new test, `test_no_page_links_to_a_redirect_source`, enforces what only the sitemap checked before. 96 pages, 1342 site tests, 276 integration tests, every drift gate clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…blocks that were prose Three passes over the page, each driven by looking at the rendered result rather than the markup. **Words.** Every phrase that gestured at meaning instead of stating it is replaced with the thing it meant: "add what it cannot see" -> "add GDP, inflation and interest rates"; "Anyone can publish a forecast. Almost nobody publishes the reasons not to trust it." -> "Every model here says where it fails"; "a verdict written against its own artifacts" -> "a plain verdict, checked against the files that produced it"; "statute as code over enhanced survey microdata" -> "tax and benefit law written as code, run over survey data"; "conditioning baseline" -> "the path FRB/US starts from"; "read order of magnitude" -> "treat the size as roughly right, not exactly right"; a dozen more of the same kind. Nothing was softened — "no skill", "inputs, not results" and "a benchmark, not a validation" stay, because they are precise rather than vague. **Alignment.** Rendered at 1440 the page had three left edges: a text measure wrongly applied to cards capped sections 01 and 03 at 980px while 02 ran the full band, and 04's tabs and tables were centred, inherited from /economy where that block is the whole page. One edge now. The measure moved from the cards to the text inside them, which also stops the core model's card rendering narrower than the six extension cards below it — the inverse of the hierarchy the section states. **Structure for what was prose.** Four blocks were the same defect: content with shape flattened into arrow-prefixed sentences. - The three evidence grades are a ladder now, strongest first, with a filled step mark (3/2/1 bars) so the direction is visible before a word is read. - The worked example is set as arithmetic: one line per figure, amounts right-aligned, a rule above the total, and the identity in mono — 50,000 − 7,486 − 2,994 = 39,520 — because checkability is the claim. The personal allowance is a note on the income-tax line, not a fourth row, since it is why that line reads £7,486 rather than a deduction from net. Beside it, at equal weight, the population panel: same run, same code, but a national total cannot be checked line by line. - Each join is drawn as its chain — what goes in, which models, what comes out — with what-moves / result / caveat keyed beneath, and the og+microsim chain terminates in a dashed "No scored result" node: the caveat is part of the diagram, not a hedge after it. - The one scored period is drawn as what it is: the 68% band to scale, the forecast tick (2.68%) above, the outturn tick (2.80%) below, and "one period inside a band is not a track record" on a warning rule at the same measure as the hit. All nine caveats survive; three are more prominent than before. style.css grows by new index-only classes (30 names, checked against every page) plus .home-page scopes; nothing that existed at HEAD is removed or altered, and .pipeline / .qa-facts keep their definitions for the pages that share them. Verified rendered at 1440 light and dark, 390, and 320; no horizontal scroll at any of them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A fresh composition rather than another patch. What changed and why: **The headline now carries the differentiator.** "Two economies, seven open models. Each one says how far to trust it." — the honesty is in the first sentence a reader sees, not discovered three bands down. The kicker gains "UK & US" so both countries are claimed immediately. **The claim and the evidence share one band.** "Every model here says where it fails" now sits directly above a two-column proof: the three verdict grades on the left (exact / a benchmark, not a validation / no skill, with the filled step marks), and the one scored period on the right — forecast 2.68% and outturn 2.80% drawn on the 68% band, with "one period inside a band is not a track record" on its warning rule. Saying it and showing it were two screens apart; now they are one glance. **The extensions became rows, not cards.** Six models as aligned rows — identity and grade | what it adds | the evidence and its limit — so the eye can scan down one column to compare what each model contributes, and down the last to compare how far each is trusted. The 2x3 card grid made every comparison a diagonal. **The record band became "the economy now."** With the scored period promoted to the proof band, the bottom tables are what they really are: the latest data beside what the models expect next, UK and US, with the no-US-forecaster statement kept in the intro. Everything else holds: the arithmetic panel, the join chains with their keyed what-moves/result/caveat rows, the generated hero-fan and glance blocks byte-intact, every number and caveat from the content contract, one left edge throughout. Verified rendered at 1440 light and dark and, under true device emulation, at 390 and 320 — scrollWidth equals viewport at both, no clipped text. (A plain --window-size=390 headless render shows clipping, but that is Chrome's ~500px minimum window width clipping the screenshot, not the page: the same page measures 390/390 under Emulation.setDeviceMetricsOverride. Noted here so the next person does not chase it.) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…y the model against its artifacts The user asked two things of the OBR emulator: is it reliable, and do its pages show how it combines with the microsimulation. A fresh audit recomputed every headline number live rather than quoting docs. **Reliability, re-verified.** Anchored GDP 0.154% / consumption 0.253% MAPE against the March 2026 EFO, confirmed one test not two (identical errors in £m across all twelve quarters). Free-running GDP 4.484%, 6 of 11 computed variables in band, 10 of 21 passthrough. The household lever re-run live: 0.17 year-one building to ~0.40 by q12 against the OBR's 0.3-decaying — slightly above the review's figures because of the March re-anchoring, and the model repo's README now carries the current profile with the November vintage as context. The other three levers remain as gated: identity, dead, non-convergent. Both slow suites green (175 passed). **One unsupported number fixed.** The overview said the 2027Q4 reform effect was −£0.40bn; the artifacts give −0.0574% x £730.6bn = −£0.42bn. **The bridge is now documented where the model lives.** /obr#pe-bridge carries the full walkthrough — the chain, the mechanism (HHDI_ADDFACTOR = minus the annual costing, converted to £m per quarter and held), the worked 1p case (£6.46bn static, GDP −0.020% impact to −0.057% by 2027Q4), and the runnable command verified against cli.py. The methodology page gains step 6 for the bridge; the code page's example is the real command. The caveats ride at the same prominence: the impact multiplier is ~1 by construction against the OBR's published 0.6, so the macro feedback is an upper bound; corporation tax is refused (not household-borne, and the direct lever never converges); no US bridge exists. Cross-linked to /models#compose and the landing page's join. Model repo doc fixes ride separately in that repo's tree: the scorecard narrative quoted superseded 14.76% household-income figures against a table reading 6.27%, now dated to their vintage. 96 pages, 1342 site tests, every drift gate clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…g silent Two strands, both on the OBR model. ## Site The overview tab now uses the same scrollytelling layout as methodology and code: narrative steps left, a sticky panel right that swaps with the step in view, mirrored into meth-inline blocks for the <900px layout where the sticky column is hidden. Steps keep the ids #pe-bridge, #what and #answers, so /obr#pe-bridge still resolves, and the generated obr-current-outlook chart block is untouched so make_current_outlooks.py still finds it by marker. Step/panel sync is the code page's version, which pairs by data-step rather than array position. Also updates the pinned OBR revision to 81839a8 (PolicyEngine/ obr-macroeconomic-model#37) and the TCPRO copy that went with it: corporation tax converges slowly now rather than not at all. Two stale "never converges" claims are fixed in papers/obr-macro and obr/methodology, which the first pass missed. ## Adapter caveats Three things the model knew and the payload did not say. An MCP caller reads JSON; it never sees a docstring, a code comment, or the website, which is the stated reason the other levers carry caveats at all. - investment_closure_plateau_fraction was promised by TCPRO's own caveat text but dropped: it lived on the frame's .attrs, which does not survive the trip over MCP. Now surfaced (0.3985 for TCPRO, matching the documented ~40% of plateau; None where there is no closure). - CGG now carries a caveat, and the test that asserted it must not is rewritten. The exemption was justified by the "~1 by construction" reading being documented on the site -- but the site is exactly what a JSON consumer never sees. Measured: +1250/qtr moves GDP +1.2500bn/qtr flat for twelve quarters, an impact multiplier of exactly 1.0000, with consumption moving <0.1% of the shock and investment zero, against the OBR's published 0.6 (July 2015 EFO, Box 3.2). That is the largest published-vs-modelled gap of any lever here, so it was the last one that should have shipped silent. - obr_shock and score_reform now carry a solver block, and the solver warning is promoted into score_reform's caveats list. Backed by PolicyEngine/obr-macroeconomic-model#38: on the 1p basic-rate path all 20 quarters exit the Gauss-Seidel stall break in both the baseline and the shocked solve, and the reported delta is the difference of two non-converged solves. The reader is defensive -- an older pinned obr-macro-model has no such attrs and returns None rather than raising. Site suite 1342 passed; integration 276 passed. NOTE: the pin still points at 81839a8, which predates #38. It needs bumping to the new main SHA once #38 lands, or the solver block stays None in production. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lcj9DDqam9KmVCfhEdnJcJ
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
PolicyEngine/obr-macroeconomic-model#38 landed as 1ed742c. Without this the adapter's new solver block reads attrs the pinned build does not set, so it degrades to None and reports nothing -- which is exactly the silence #38 existed to end. Verified against the pinned build: obr_shock('CGG', 1250, periods=12) returns solver.converged False with all 12 quarters listed as non-converged, and the warning present. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Lcj9DDqam9KmVCfhEdnJcJ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two strands, both on the OBR model. Includes the four earlier redesign commits already on this branch.
Site
The overview tab is now two-sided, matching methodology and code: narrative steps left, a sticky panel right that swaps with the step in view.
HHDI_ADDFACTOR, spreading, chain, what's refusedPanels are mirrored into
meth-inlineblocks for the <900px layout where the sticky column is hidden. Steps keep the ids#pe-bridge,#what,#answersso/obr#pe-bridgestill resolves; the generatedobr-current-outlookblock is untouched somake_current_outlooks.pystill finds it by marker; step/panel sync is the code page's version, which pairs bydata-steprather than array position.Also bumps the pinned OBR revision to
81839a8(PolicyEngine/obr-macroeconomic-model#37) with the TCPRO copy that goes with it, and fixes two stale "never converges" claims the first pass missed (papers/obr-macro,obr/methodology).Adapter caveats
Three things the model knew and the payload did not say. An MCP caller reads JSON — it never sees a docstring, a code comment, or the website, which is the stated reason the other levers carry caveats at all.
investment_closure_plateau_fractionwas promised by TCPRO's own caveat text but dropped: it lived on the frame's.attrs, which does not survive the trip over MCP. Now surfaced —0.3985for TCPRO, matching the documented ~40% of plateau;Nonewhere there is no closure.CGGnow carries a caveat, and the test asserting it must not is rewritten. The exemption was justified by the "~1 by construction" reading being documented on the site — but the site is exactly what a JSON consumer never sees. Measured:+1250/qtrmoves GDP+1.2500bn/qtrflat for twelve quarters — an impact multiplier of exactly 1.0000, with consumption moving <0.1% of the shock and investment zero — against the OBR's published 0.6 (July 2015 EFO, Box 3.2). Largest published-vs-modelled gap of any lever here, and the last one that should have shipped silent.obr_shockandscore_reformnow carry asolverblock, and the solver warning is promoted intoscore_reform'scaveats. Backed by Report solver convergence on reform results instead of discarding it obr-macroeconomic-model#38: on the 1p basic-rate path all 20 quarters exit the Gauss–Seidel stall break in both the baseline and the shocked solve, so the reported delta is the difference of two non-converged solves. The reader is defensive — an older pinnedobr-macro-modelhas no such attrs and returnsNonerather than raising.Testing
Site suite 1342 passed; integration 276 passed.
The pin still points at
81839a8, which predates obr-macroeconomic-model#38. Once #38 lands, the pin inintegration/pyproject.tomlandintegration/modal_app.pyneeds bumping to the newmainSHA — otherwise thesolverblock returnsNonein production.Known issue, not addressed here
The published worked-example figure −0.057% (−£0.42bn) at 2027Q4 is wrong — the actual value is −0.0854% (−£0.585bn), identical at both the old and new pins, so it is not caused by the revision bump. It appears in four files. Given #38's finding that per-quarter values are stopping-rule artifacts, this wants a decision about how much per-quarter precision the page should claim, not just a find-and-replace.
Relatedly, the OBR's published multiplier for income tax/NIC is 0.3, not the 0.6 the page cites (0.6 is day-to-day spending and welfare). The income-tax bridge's implied multiplier builds to ~0.35–0.39 — close to the OBR. So the page currently applies CGG's
~1 by constructiondefect as a caveat on the income-tax example, where the model actually performs well.🤖 Generated with Claude Code
https://claude.ai/code/session_01Lcj9DDqam9KmVCfhEdnJcJ