Skip to content

Evidence publishes refresh the mutable root convenience copies (populace_us_2024.h5 at repo root) — decide whether the evidence tier should skip them #753

Description

@MaxGhenis

Surfaced by sol's merge adjudication of #674 (evidence tier, #506) as a post-merge caution, not a defect: publish_release writes the manifest-declared root artifacts (the H5/NPZ at their repo-root paths) in the main-branch commit for every non-tag-only publish. That is pre-existing behavior — a certified --no-latest non-default publish already overwrites the root copies today — so an --evidence publish with the default pointer update will likewise replace the root populace_us_2024.h5 with the evidence artifact.

Why it matters more at the evidence tier: a naive hf_hub_download("policyengine/populace-us", "populace_us_2024.h5") with no revision reads main's root copy and would receive an artifact whose gates failed, without seeing the tier marker that lives in the release manifest and the -evidence- tag. Every contract-aware path is unaffected: latest.json never moves, the registry/loader pin artifacts by revision (the release tag), and latest-evidence.json paths point under releases/<id>/.

Options for the owner:

  1. Structural: at the evidence tier, upload root artifacts only in the immutable tag commit and omit them from the main-branch commit (evidence H5 reachable by tag + manifest revision only). ~10 lines in _publish_atomic plus a test asserting the evidence main commit carries no root artifact paths.
  2. Operational: publish evidence with --tag-only-like semantics for roots but still move latest-evidence.json (needs a new flag combination; today --tag-only also suppresses the pointer).
  3. Status quo: document that root copies are "whatever published last" (already the non-default semantics) and rely on consumers pinning revisions.

Recommend (1): it keeps the tier's "never mistakable for certified" guarantee structural for the one remaining unpinned read path. Worth deciding before the first evidence publish.

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions