Evidence-tier publishing: contract sibling, latest-evidence.json, builder --evidence-release (#506)

[MaxGhenis]

Implements the #506 design (owner directive: the best available artifact should be publishable at all times WITHOUT weakening certified gating), generalizing the #490/#497 register pattern — adjudicate, record, carry the caveat in the artifact. Leaves #506 open for the owner to close after the first evidence publish (the current best dense artifact).

The tier, structurally

• validate_evidence_release_dir (microcosm-data): a sibling of validate_release_dir, not a relaxation — same required files, same shape/provenance/hash/cross-manifest checks, plus a mandatory non-empty known_failures block: every recorded gate failure verbatim, each with an owner issue ref. Gate-verdict requirements (US critical-target fit, source-coverage gate.passed) are exactly what the tier records instead of enforcing. Dirty-git, hash mismatches, and build-id mismatches still refuse: the tier relaxes verdicts, never auditability.
• Mutual exclusion is structural, both directions. Evidence manifests declare schema_version: "1-evidence" + tier: "evidence"; the certified contract rejects that schema marker no matter which gates failed, and a certified-shape manifest (no tier, no known_failures, schema 1) fails the evidence contract. validate_release_dir itself is untouched.
• Scope: national-default releases only. Local-area roles (Release contract needs dataset-role classes: national-default checks reject valid non-default local-area artifacts #398) and UK exact-k releases (One country-agnostic gate battery: phased evaluation, unconditional reporting, per-country gate selection (#578) #611 gate battery) are refused by name — no evidence semantics have been adjudicated for them.

Publish path

• publish_release(evidence=True) validates via the evidence contract, creates the immutable tag exactly as today, and moves only a new latest-evidence.json pointer (payload mirrors latest_pointer_payload + tier). The certified latest.json path never appears in an evidence commit — pointer isolation is pinned in both directions by tests.
• latest_evidence_release() is the consumer; latest_release() now refuses a pointer payload naming a foreign tier (defense if an evidence payload ever landed in latest.json).
• microcosm-publish-release --evidence; Slack alerts label the tier so they can never read as a certified release announcement.

Builder --evidence-release

• With the flag absent, behavior is bit-identical to main: all conversion points keep the exact certified statements (the batched pre-export raise with the CG tail transfer: field-local donor quarantine + mass-conserving own-tail stratum + E01000 receipt (#567) #568 weight-evidence sidecar, the reform-smoke raise, the take-up-contract raise), and the only refactors are string-identical (list-then-join, extracted _default_release_id).
• With the flag: the Fold SSI take-up gates into the batched terminal gates so failed runs keep their evidence #548 terminal accumulator, reform smoke, and take-up contract convert from raise to record; the run continues through the H5 write and manifests, and _evidence_release_manifest_fields stamps the evidence schema marker + known_failures (raises on an empty record — the flag is structurally incapable of minting a certified-shape manifest). Preflight and mid-build source-stage gates still abort in both modes.
• Owners are mandatory: each recorded failure must match the standing US_EVIDENCE_FAILURE_OWNERS register (the two Evidence-tier publishing: ship the best available artifact at all times, with its failures recorded in the manifest (dense first) #506-adjudicated dense families: SOI Table 1.4 dollar fit → PUF donor uprating overstates mean-reverting distribution series: non_sch_d at 2.39x the SOI concept #487, QRF tail concentration → QRF amount forests lose donor design weights after the n-of-n bootstrap: extreme-tail draws are a seed lottery #481/PUF donor uprating overstates mean-reverting distribution series: non_sch_d at 2.39x the SOI concept #487) or a --evidence-failure-owners per-run JSON — an unowned failure refuses the export, at the earliest conversion point.
• An all-green run under the flag is refused ("rerun without --evidence-release"): the tier exists to carry failures, and a certified-quality artifact must certify.
• Release ids: auto-ids carry the -evidence- segment; certified builds refuse ids that squat it; incompatible with --exact-k (the ladder lane keeps its tag-only contract).

Tests

49 new tests across the three suites (19 contract, 13 release/publish, 2 CLI guard, 11 builder + the #568 AST guard now also covering the evidence branch shape). Full workspace suite green (uv run pytest: exit 0 on this branch), ruff clean.

Notes for the merge gate

• Ported from the pre-rename populace tree (branch evidence-tier-506, commit 9012391d) onto microcosm paths/namespace.
• Merged origin/main (d69131a, post-Replacement scorecard: head-to-head yardstick and the incumbent's score #748) in 03a908a1 after the branch drifted 425 commits; three hand-resolved hunks, all unions: publish_cli.py (main's --allow-missing-staging + --evidence), test_contract.py (main's schema-4 battery tests, then the evidence section), and the builder's _parse_args combo errors. Follow-ups: the Evidence-tier publishing: ship the best available artifact at all times, with its failures recorded in the manifest (dense first) #506 AST guard now inspects _main (main's Staging telemetry silently disabled since 2026-07-02: empty POPULACE_STAGING_REPO_ID defeats the on-by-default staging repo #563 wrapped main() around _main()); ruff format whitespace on two files. Suites green on the merged tree (522 passed), ruff check . clean.
• Main's new _staging_undelivered publish guard applies to evidence publishes too (the build manifest is shared across tiers): an evidence build that meant to stage and delivered nothing is refused the same way, and --allow-missing-staging overrides it the same way. Evidence builds stage by default like certified ones.
• gates.calibration in the build manifest stays the battery-only verdict in both modes; the release manifest's known_failures is the authoritative full terminal record (a superset, bound to both recorded lists and the recomputed critical-fit verdicts).
• Cross-family review: two sol rounds done and adjudicated (see the review-trail comment below); a third round on the merge delta is running and will be appended to that trail.
• Do not merge from this lane: the OWNER session does the final merge gate and the first evidence publish. Evidence artifacts never feed pe.py certification.

🤖 Generated with Claude Code

[MaxGhenis]

Cross-family review trail (sol, 2 rounds)

Both rounds ran gpt-5.6-sol (ultra effort, read-only) against the committed diff only, with an adversarial brief enumerating the tier invariants. Every finding was adjudicated; fixes landed as their own commits so the trail is inspectable.

Round 1 (on the initial three commits) → fixed in 7032f3ad

1. HIGH, fixed — pointer isolation bypassable via manifest-declared root artifacts (an artifact entry whose path IS latest.json). Both pointer paths now reserved against root artifacts on both tiers, tested in both directions.
2. HIGH, fixed — known_failures unbound from recorded gate results (fabrication/softening possible). Bound in round 1 to gates.calibration.failures + recomputed critical-fit breaches.
3. MEDIUM, fixed — generic release ids (acme-evidence-build) deactivated the US-specific requirements; the evidence contract now requires the populace-us- prefix.
4. MEDIUM, adjudicated no-change — certified builds refusing -evidence- ids is the intentional reserved-namespace guard (a formerly-accepted hypothetical id now refuses; disclosed in the PR body, pinned by test).
5. MEDIUM, fixed — latest_release accepted explicit tier: certified/null payloads; now ANY tier field in latest.json refuses.
6. LOW, accepted — LatestPointer gained a defaulted tier field (additive API change; in the changelog fragment).
7. COVERAGE, fixed — main()-flow conversions were unpinned; added the AST control-flow guard.
8. PLAUSIBLE, fixed — an evidence attempt refused pre-export for an unowned failure lost the CG tail transfer: field-local donor quarantine + mass-conserving own-tail stratum + E01000 receipt (#567) #568 weight-evidence sidecar; refusals now fall through to the certified failed-run path (sidecar included) before re-raising.

Round 2 (on the round-1 fixes) → fixed in 3850350c

1. HIGH, fixed — ./latest.json dodged the raw-string reserved-path check; root-artifact paths must now be canonical clean relative POSIX form, and all release-dir uploads must be bare file names (extra_files traversal included).
2. HIGH, fixed — binding was fail-open when gates.calibration was deleted; both locally checkable records (gates.calibration.failures, diagnostics build.release_gates.failures) are now REQUIRED and shape-checked, then bound verbatim.
3. HIGH, fixed — us_source_coverage gate.failures were relaxed but unbound; now each must ride within a known_failures entry, and the builder records them (owner-checked) at the evidence tier.
4. MEDIUM, fixed — acknowledgment matching hardened to name-delimited against the diagnostics' own target names (ctc_amount no longer satisfiable by an actc_amount entry). Residual caveat, documented in the helper docstring: the binding guarantees each recorded failure is named with an owner; prose sentiment around it is for human review, as in the Adjudicate the medical-dental publish tolerance to 0.25 pending the loss-contract alignment #490 register pattern.
5. MEDIUM, adjudicated no-change — a _TerminalBatchTelemetry crash line appended after the batched owner check exports the H5 before the final owner check refuses manifests; consistent with the CG tail transfer: field-local donor quarantine + mass-conserving own-tail stratum + E01000 receipt (#567) #568 late-gate doctrine (weights retained in the written dataset), now documented at the conversion site.
6. MEDIUM, fixed — the AST guard had mutation false-passes (polarity flip, evidence_known_failures=None, unreachable refusal); now pins exact guard forms, the exact refusal guard chain, the forwarded Name, and all five owner-resolution sites.
7. PLAUSIBLE, fixed — extra_files traversal covered by the bare-file-name rule above.

Round-2 'held' list (attacked, no hole): verbatim-binding on well-formed records, non-empty/owned known_failures, exact-path reservation, in-dir latest.json uploads staying under releases/<id>/, tier-field rejection, certified byte-identity of the three raise sites and sidecar branch, single-sidecar refusal ordering, final owner recheck closing late unowned failures, and the id-scope triple (prefix + UK exact-k + role refusals).

🤖 Generated with Claude Code

[MaxGhenis]

Round 3 (on the merge of origin/main d69131a → 03a908a1)

Brief: does the #506 tier survive main's 425 commits — auditability parity after main's 22 contract.py commits, the conversion sites under the reworked terminal flow (early_terminal_gate_failures, _TerminalBatchTelemetry, the main()→_main() wrapper), publish-side interaction with the new _staging_undelivered guard, fidelity of the three hand-resolved hunks, and test counts.

No HIGH/MEDIUM/LOW findings. Held: every US national-default certified check is still enforced by the evidence sibling (main's contract commits are UK-only except the diagnostics schema bump, which flows through the shared helper); UK exact-k / non-default roles still refused; of main's 71 _main raises, 68 match exactly and the other 3 are the intended conversions with identical certified guards, messages, and side-effect order; early failures still fold into terminal_gate_failures; no post-accumulator verdict escapes it; the five owner-resolution sites are ordered; release.py untouched by main and every evidence path still selects only latest-evidence.json; the three conflict hunks are exact unions with no drops or duplicates; 528 tests collected (320 data + 208 builder), ruff 0 diagnostics.

One PLAUSIBLE, adjudicated out of scope → #751: main's _ACTIVE_TELEMETRY global is only reset inside _staging_telemetry(), so in a multi-invocation process an early exit in a later main() call (argparse/dirty-worktree exits on main already; --evidence-failure-owners validation adds one more) would .fail() the prior run's telemetry — pre-existing class from #563, unreachable for one-process-per-run CLI usage, fix belongs with that lifecycle.

🤖 Generated with Claude Code