From 3ea6b614d6c687eb902efacab83db797c3452fdb Mon Sep 17 00:00:00 2001 From: vahid-ahmadi Date: Wed, 12 Aug 2026 14:09:24 +0100 Subject: [PATCH] Fix what five page-level audits found, including two self-contradictions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The five section audits killed when the machine slept were rerun. Their findings, worst first. /forecasts headlined an accuracy number, which is the one thing its own generator says it must never do. The record band's third tile read "0.12pp — latest absolute error", level with the round and scored counts, while score.py's module docstring argues that "a track record that reports an impressive figure derived from one observation is worse than an empty one", and the disclaimer saying so sat below the tile. The tile now shows the next scored period. The error still appears on the scored card, beside its benchmarks. The same page called its one favourable result "beats naive" using the driftless walk that its own section 06 calls too weak for a trending series. The footer now names the benchmark: "beats driftless-naive error". frb-us/index promised a "shock explorer on this page". There is none, on that page or anywhere on the site — the page contains no form, input or button. A reader was told to do something impossible. /economy carried the only hand-typed live number on the page, and it had already rotted: "the model expects it back above 3% by Q4", from the 2026-07-21 round, against a current round putting 2026Q4 at 2.94% and an inflation topic one click away saying not until 2027Q2. It now describes the note as the dated artifact it is and points at the live view. svar/index cited the 49-origin rolling evaluation as a validation credential. That study is precisely what found the model no better than naive: against a random walk with drift, UK GDP is not statistically distinguishable (p=0.38-0.67). The page published a GDP fan with no qualifier. Both now sit in the fact. obr/index presented 0.15% anchored MAPE as validation. Anchored accuracy is a by-construction invariant — the page's own validation page says so — and free-running the same equations miss by 5.75%. Both now stated. Every equation and code block on fourteen pages was invisible to screen readers. .meth-inline and .scrolly-sticky are a responsive pair, exactly one visible per viewport, and both carried aria-hidden="true" — so in every viewport the only visible copy was hidden from assistive technology. On pe/code and define/code, whose entire subject is the commands, not one was exposed. 58 blocks across 14 pages now are; the copy buttons in the hidden aside are no longer focusable. The sticky column blanked through step 1 on five pages, and that one is mine: my simulation of the data-step pairing skipped the active === 0 case, which is exactly the case that fails. Panels now pin from the first available step. models/index called all 284 FRB/US equations behavioural; the paper says 120 behavioural and 164 identities. It counted seven model classes where there are six — obr-macro and frb-us share one. Its HMRC link said June 2025 and pointed at the January bulletin. And it attached "machine precision" to the ~1e-8 cross-validation figure, which is solver tolerance, not the 5.6e-17 tracking invariant. /connect said "everything else here runs hosted"; the three define_* tools return run instructions, as the unlicensed upstream is never hosted. /models#score landed on a closed accordion because the id sits on the
element, which browsers do not auto-expand — breaking the primary "score a reform" journey from three high-traffic links. reveal.js now opens a details element a fragment targets. olg/validation published debt-to-GDP as 95.1% for May 2026. The committed store says May is 94.8 and June 94.9, and the site's own economy page says 94.9. Corrected, with the deviation sign flipped to match. And the DEFINE clean-room status was stale everywhere, which is my doing: I merged §3.3.1 and §3.3.2 today and updated neither the site nor the registry. Four surfaces said "§3.2, 23 equations, three defects" against a reality of 118 equations and some thirty-five. The §3.3.2 blocker — six untabulated parameters and two variables defined nowhere, so the section as published cannot be simulated forward — was published nowhere at all, while the same pages call the clean room the path to hosting. Now stated. One audit finding was wrong and is not acted on: define/validation cites upstream issue DEFINE_UK_1.1#1 for the emissions divergence, flagged as unsupported. The issue exists, is open, and its title matches the finding exactly. Verified before deleting anything. Twelve gates, 1038 site tests, 274 integration tests. Co-Authored-By: Claude Opus 5 (1M context) --- connect/index.html | 2 +- define/code/index.html | 24 ++++++++----- define/methodology/index.html | 12 +++---- economy/index.html | 5 ++- forecasts/index.html | 4 +-- forecasts/score.py | 26 ++++++++++---- frb-us/code/index.html | 20 +++++++---- frb-us/index.html | 4 +-- frb-us/methodology/index.html | 12 +++---- .../src/policyengine_macro/capabilities.py | 3 +- models/index.html | 6 ++-- obr/code/index.html | 20 +++++++---- obr/index.html | 2 +- obr/methodology/index.html | 10 +++--- olg/code/index.html | 34 +++++++++++-------- olg/methodology/index.html | 22 +++++++----- olg/validation/index.html | 2 +- pe/code/index.html | 20 +++++++---- pe/methodology/index.html | 14 +++++--- reports/define-uk-replication/index.html | 14 ++------ reveal.js | 25 ++++++++++++++ svar/code/index.html | 20 +++++++---- svar/index.html | 2 +- svar/methodology/index.html | 10 +++--- us-hank/code/index.html | 18 ++++++---- us-hank/methodology/index.html | 16 ++++++--- 26 files changed, 220 insertions(+), 127 deletions(-) diff --git a/connect/index.html b/connect/index.html index 512153aa..2c3a88ef 100644 --- a/connect/index.html +++ b/connect/index.html @@ -372,7 +372,7 @@

Connect in three steps.

OG-UK long-run scoring (score_reform with model='og') and dynamic_reform_impact take tens of minutes — the hosted server points you to the local CLI. - Everything else here runs hosted.

+ The three define_* tools likewise return run instructions rather than results when called on the hosted server — the unlicensed upstream is never hosted. Everything else here runs hosted.

diff --git a/define/code/index.html b/define/code/index.html index c8bc13fd..362c74eb 100644 --- a/define/code/index.html +++ b/define/code/index.html @@ -103,7 +103,7 @@

Where to read the results

2

CLI scenario runs

Deltas from the terminal

The CLI lists the curated scenarios and runs any of them locally: pe-macro define-scenarios to browse, pe-macro define-scenario to produce a scenario's deltas vs baseline — real GDP, emissions, unemployment, real consumption. Deltas only, never levels.

-