Mode-3 case schema + UK battery: UKMOD/EUROMOD lane groundwork

[vahid-ahmadi]

Part of #41 — the schema/battery half of the UKMOD/EUROMOD mode-3 lane. Designed per #5 as the single case-level schema shared with the TAXSIM lane: country on the case and oracle (ukmod | taxsim) on the result row keep it engine-agnostic, so mode 3 gets one schema, not two.

What's here

• sources/ukmod-cases/SCHEMA.md — the case-diff contract. A case = {case_id, description, policy_year, country, household, expected_focus, rationale} with an engine-neutral, closed household input vocabulary (people with ages/incomes/disability + carer flags, benefit-unit structure, region/tenure/rent) that each lane's connector maps to its engines' variables. A result row = {case_id, variable, pe_value, oracle_value, oracle, engine_version, oracle_version, computed_at, abs_diff, classification}. Classification is a fail-loud closed set — match_exact / match_within_tolerance / pe_gap / oracle_difference / policy_scope_mismatch / rounding / unclassified — with oracle_difference and rounding reachable only by adjudication with a writeup. Tolerances per variable class: currency £0.01/week (0.52/year annual comparisons), booleans exact.
• sources/ukmod-cases/battery/cases.json — 14 curated UK hypothetical households, inputs + expected focus only (no fabricated expected values; both sides of every comparison come from engine runs later). Coverage: single unemployed on UC; lone parent + 2 children with LHA rent; two-child limit binding with a pre-April-2017 protected child; London benefit cap; 55% taper + work allowance; minimum income floor; Pension Credit single + couple; mixed-age couple routed to UC; Scottish bands + Scottish Child Payment; HICBC at the £70k mid-taper; NI threshold edges (£12,570 / £50,270) with salary sacrifice; carer CA + UC carer element; £100k personal-allowance taper. Round, auditable inputs with a rationale per case.
• scorecard_db/case_diffs.py — CaseSpec / CaseResult dataclasses in the models.py fail-loud style (closed enums raise on unknown values; benefit units must partition people; owner-occupiers carry no rent; abs_diff must reconcile), the battery loader, and classify() — the first-pass classifier whose above-tolerance default is unclassified, never a flattering bucket.
• tests/test_case_schema.py — battery loads/validates, unique country-prefixed ids, inputs-only invariant, focus coverage, classifier and result-row edge cases (28 tests).
• data/lanes.json (+ app/public/data mirror) — ukmod-cases advanced registered → cataloged.

Out of scope (remaining for #41)

The JRC connector run itself (needs the UKMOD environment): vocabulary→UKMOD input mapping, executing the battery on both engines, appending CaseResult rows, and publishing the miss table.

Verification

• PYTHONPATH=. uv run --with pytest pytest tests/ -q — 156 passed, 4 skipped
• uvx ruff format --check scorecard_db tests pipeline — clean
• data/lanes.json regenerated with the feed's exact serialization; app mirror byte-identical

🤖 Generated with Claude Code

[MaxGhenis]

Dual-gate review: NEEDS-FIXES. Blockers: (1) arbitrary misses can be blessed — match_within_tolerance/rounding/oracle_difference validate with empty annotations and no tolerance or adjudication writeup, contradicting the module contract and SCHEMA.md; (2) closed-schema gaps — benefit-unit adults/children and expected_focus aren't required to be lists, so one-character strings validate; (3) the two-child-limit case can't exercise its rationale — policy year 2026 begins 6 April, the day the limit ended, and exempting the eldest (already within the first two children) never tests a third-child exemption. Verified: no dependency on #48.

[vahid-ahmadi]

All three blockers fixed in 838cc45.

#	Blocker	Fix
1	match_within_tolerance/rounding/oracle_difference validated with empty annotations and no tolerance or writeup	CaseResult grows a tolerance field, required on (and only on) match_within_tolerance rows with 0 < abs_diff <= tolerance enforced — stored per row so a tolerance-table change can never silently re-bless history. oracle_difference/rounding are now ADJUDICATED_ONLY: they raise without a non-empty annotations writeup. annotations itself must be a list of non-empty strings. SCHEMA.md documents all of it
2	Benefit-unit adults/children and expected_focus accepted strings (iterate as characters)	Both now require actual lists — person-id strings for unit members, variable names for expected_focus; a bare string raises. Pinned by tests either way
3	The 2026 two-child-limit case could not exercise its rationale (limit ended 6 April 2026; the "exempt" eldest was already within the first two elements)	Replaced with an honest pair: uk-uc-two-child-limit-multiple-birth at policy year 2025 (limit in force), all three children post-April-2017 and the multiple-birth exception sitting on the third child, so the exemption strictly changes entitlement (3 elements vs 2); and uk-uc-two-child-limit-abolished — the same family in 2026, where an engine still applying the limit under-pays exactly one element. test_two_child_limit_pair pins the year split, post-2017 DOBs, and the shared-DOB construction

Tests: test_case_schema.py 25 → 37 (empty-annotation rows fail for each of the three statuses; string adults/children/expected_focus fail; tolerance bounds at both edges; zero-diff can never be match_within_tolerance). Full suite 164 passed / 4 skipped; ruff format --check clean.

Note: the battery now spans policy years {2025, 2026} — test_uk_battery_shape says why. Case ids are safe to rename since no CaseResult rows reference the old id (the connector run is still pending the UKMOD environment).

[DTrim99]

Review — Mode-3 case schema + UK battery

Disciplined contract PR. The closed vocabularies are enforced in code, not just SCHEMA.md prose: unknown oracle / classification / country / person-key / household-key / case-key all raise (case_diffs.py:223-291, 346-402, 447-471) and are tested (test_case_schema.py:1119-1179, 1250-1256). The "single schema shared with the TAXSIM lane" claim holds — there's no divergent taxsim-cases schema in the tree, so this is the one definition both lanes will share. Battery cases are inputs-only, each with a rationale. No blocking issues.

Should address

• classify() → CaseResult tolerance seam is untested end-to-end (case_diffs.py:405-430 vs 336-397). classify returns MATCH_WITHIN_TOLERANCE but not the tolerance it used; the caller must re-derive DEFAULT_TOLERANCES[variable_class] and thread it into CaseResult.tolerance, which re-validates. Nothing wires classify → CaseResult, so a caller passing the wrong tolerance/variable_class is uncaught. Either add one integration test or have classify also return the applied tolerance — worth doing before Official-calculator oracles (mode 3): closed-set extension, dated-reading provenance, calculator work list #64's connector lane relies on it.
• variable_class never persisted on CaseResult (case_diffs.py:329-344) — a stored row can't be re-classified/audited for which tolerance rule applied except via the free-text variable name.

Suggestions

• No schema_version on the battery/result — a foundation contract that Official-calculator oracles (mode 3): closed-set extension, dated-reading provenance, calculator work list #64 stacks on benefits from an explicit version for forward migration.
• date_of_birth is validated by shape only (len==10, dashes at 4/7) — "2026-13-40" passes. Ages are the load-bearing field, so minor.

Reviewed with Claude Code assistance.

[vahid-ahmadi]

All four review items addressed in a1d57d7:

Item	Fix
classify → CaseResult seam untested	CaseResult.from_classification() is now the one wiring path: it derives abs_diff, threads the exact tolerance classify judged against (only onto match_within_tolerance rows), and persists the variable class — no caller re-derives DEFAULT_TOLERANCES[...] by hand. Six end-to-end seam tests, including a custom-table case proving the stored tolerance is the one actually applied.
variable_class not persisted	Added to CaseResult (closed vocabulary, validated), set by the constructor path, documented in SCHEMA.md.
No schema_version	SCHEMA_VERSION = 1 on the battery file (missing/wrong version raises in load_battery) and on every result row (foreign version raises) — #64's connector lane stacks on an explicit contract version.
date_of_birth shape-only	Now parsed with date.fromisoformat; 2026-13-40, 0000-00-00, and 2025-02-30 all fail, 1996-02-29 passes.

Suite: 176 passed / 4 skipped; ruff clean.

[DTrim99]

Re-review — all addressed ✅

Went back through my prior notes against the new commits; everything's resolved:

• classify→CaseResult tolerance seam — CaseResult.from_classification(...) is now the single wiring path (threads the exact tolerance classify judged against), covered end-to-end by TestClassifyToResultSeam.
• variable_class persisted on CaseResult and round-tripped by the seam.
• schema_version on the battery + every result (loader rejects mismatch), and date_of_birth now real-parsed via date.fromisoformat so 2026-13-40/2025-02-30 fail.

Nice — no remaining concerns from my side.

[DTrim99]

Re-reviewed — all prior findings are addressed with tests (the classify→CaseResult from_classification seam, variable_class persisted, schema_version, and real-date date_of_birth validation). Approving.

[vahid-ahmadi]

Rebased onto main (post #70–#73) — it had gone conflicting on three trivial spots: the lanes.json updated date (both copies kept identical) and two __all__ orderings in scorecard_db/__init__.py, all resolved as unions with no semantic change. Suite 290 passed / 6 skipped, format clean.

[vahid-ahmadi]

Post-#74 fix (9eff590). Verified the whole stack against the new DB-built-in-CI main and this branch was the one breakage — my own, from the earlier rebase: I hand-set lanes.json's updated to 2026-08-20, but that field is derived — sync_lane_feed() sets it from the max of the lane rows' updated_at (2026-08-19) and rewrites the file.

Invisible while the DB was committed (ingest never ran during the suite); with #74 building the DB at collection it reverts every run, so it failed both new no-drift gates and test_app_data_copies_match_committed_data (the build rewrites data/ but not the app/public/ copy, so the committed copies diverge at runtime).

Reverted to the derived value in both copies. Merged against current main locally: 295 passed / 2 skipped, no-drift clean before and after the suite. Same class of latent breakage as the PyYAML finding on #56 — old green CI was luck.

[MaxGhenis]

Re-gate round 2 — closure fixed; the seam and the epistemic wiring aren't yet

Cleared this round: unknown JSON keys and unknown enum strings now raise (the narrow closure blocker), the synthetic merge with post-#74 main is conflict-free with no committed-DB dependency, and an in-memory replay of all feed synchronizers reproduced lanes.json byte-identically — the branch is safe against the new no-drift gates as it stands.

Six findings, four blocking (each was probed, not read):

1. The classify → CaseResult seam is still bypassable. CaseResult(...) accepts a caller-supplied classification and the validator never recomputes classify() — the factory is a convention, not an enforcement. Probes accepted: 100 == 100 persisted as unclassified or pe_gap (classifier says match_exact); numeric mismatches persisted as pe_gap/policy_scope_mismatch without annotations; boolean 1 != 0 persisted as match_within_tolerance with a caller-provided tolerance. Only oracle_difference and rounding require writeups, so the adjudication claim doesn't hold either. Fix shape: the validator recomputes and compares, or construction is only possible through the classifying factory.

2. variable_class isn't actually required — still Optional[...] = None, and the shipped valid-row test itself omits it. Same class: an omitted result schema_version silently defaults to 1 rather than raising.

3. Mode-3 results bypass the repo's epistemic discipline. CaseResult carries no calibration_relationship (UKMOD is already deliberately held_out in relationships.py — case rows never record it), no executed baseline key, no case/battery digest, no bundle/run pin, no JRC connector revision, and no citable adjudication link; blank engine versions and a nonexistent case_id were accepted. There's also no case table, writer, export, or descriptive route yet, and the enum collapses raw comparison status with normative diagnosis into one field outside the action-link gate — so nothing structurally prevents match_* reading as a win or an unsupported pe_gap reading as blame. Suggest splitting status (descriptive) from diagnosis (gated, citable) exactly as external_scores/diagnoses do.

4. The two-child-limit pair doesn't test its claimed mechanism. Both the 2025 and 2026 households keep the same twins — an engine that wrongly retains the limit but rightly applies the multiple-birth exception still pays three elements, so the case can't catch the failure its rationale names; and the year-on-year delta moves ages, rates, and policy year together, so it can't isolate abolition. The registered pre_ab2025 world is the right instrument for a same-year counterfactual — cases/results just need a way to reference a registered baseline world.

5. Identity closure is local, not integrated. Probes accepted region="MARS", invented expected_focus values, NaN incomes, infinite rent, boolean-class values outside {0,1}, and an arbitrary battery schema — none of these route through uk_aliases or any case registry. The private-rent cases also don't pin one LHA world: the schema routes LHA by ITL-1 region while forbidding brma, and the Yorkshire rationale concedes the rent clears the cap only in "most" BRMAs — the two engines aren't guaranteed the same world.

6. Accuracy note on 9eff590: the current bytes are right, but sync_lane_feed() doesn't derive the updated date — it copies a caller-supplied literal, and today's literals happen to reproduce 2026-08-19. Worth either truly deriving it or documenting the literal contract, so the next caller doesn't reintroduce the drift you fixed.

47 focused tests pass and the worktree stays clean — the schema-closure work is real progress; the remaining arc is making the invariants structural rather than conventional, and wiring mode-3 into the same descriptive/gated discipline as the rest of the DB. Happy to pair on #3's table design — it's the piece with repo-wide surface.

🤖 Generated with Claude Code

[vahid-ahmadi]

Five of six addressed in 7684f64; finding 3 is deliberately left for the pairing you offered — see the bottom.

1. The seam is enforced. __post_init__ now recomputes classify() and compares. A stored classification is valid only if it equals the classifier's, or if it adjudicates a row the classifier left unclassified — drawn from ADJUDICATABLE and carrying a writeup. Every round-2 probe raises now: 100 vs 100 as unclassified/pe_gap, a numeric mismatch as policy_scope_mismatch with nothing said, and a boolean 1 vs 0 as match_within_tolerance with a caller-supplied tolerance of 100. You were right that the adjudication claim did not hold either: pe_gap and policy_scope_mismatch are now two-sided — mechanical on a null side, adjudicated and explained on a numeric-vs-numeric row. Tolerances come from DEFAULT_TOLERANCES alone; from_classification no longer takes a table, and it DOES now take annotations, which is also what made every Official-calculator oracles (mode 3): closed-set extension, dated-reading provenance, calculator work list #64 calculator call raise.

2. variable_class and schema_version are required. The validator cannot check a row without the first, and an omitted second silently reinterpreted a stored row. Blank engine/oracle versions are rejected.

3. The two-child family. Rebuilt as two same-year pairs. binding uses three SEPARATE births — you were right that twins on both sides meant an engine wrongly keeping the limit but rightly applying the exception still paid three elements and passed. abolished is a byte-identical household in the same year under current law, so binding/abolished attributes the abolition; multiple-birth differs from binding by exactly one date of birth, so it attributes the exception. Cases can now reference a registered baseline world, which is the instrument that made the same-year pair possible — thank you for pointing at pre_ab2025.

4. Identity closure. Regions are a closed per-country registry (region="MARS" raises), expected_focus is closed per country and may not repeat, NaN and infinity are rejected wherever an amount is read, a boolean-class comparison may only hold 0 or 1, the battery's schema must name a contract this repo defines, and a case baseline must already be registered. On LHA: agreed the region does not pin a world — UK private-rent cases must now pin a BRMA, and a BRMA outside its own region raises.

5. sync_lane_feed. Correct, it copies a caller-supplied literal. Documented as exactly that, with the constraint that every caller in a build must pass the same constant, so the next caller does not reintroduce the drift.

Finding 3 — taking you up on the offer to pair. The case/result table, writer, exporter and build_db step, plus the epistemic columns and the status/diagnosis split that ride on them, are the piece with repo-wide surface, and guessing at it unilaterally is how it ends up re-litigated. SCHEMA.md and the module docstring now state the open shape explicitly rather than leaving it implied. Happy whenever suits you.

Suite 306 passed / 4 skipped, ruff format clean.

[vahid-ahmadi]

Integration review finding — this branch had never met the post-#74 gates.

It forked before #74 (the database leaving git), so it carried no scorecard_db/build_db.py and still tracked data/scorecard.db as a committed binary. CI config is per-branch, so this PR was running the pre-#74 workflow: the determinism check and the no-drift gate had never executed against it at all. The green tick was a weaker check than it looked — worth knowing before reading the approval as covering them.

Current main is merged in now. The branch builds from scratch, two builds agree on content_hash, the tree is clean afterwards, and the suite goes from 287 to 325 passed because main's own tests come with it. No conflicts — main's deletion of the committed database wins over an untouched file on this side, so nothing resurrects.

Nothing in the branch's own code changed.