One-command HF publish for Cross-dataset evaluation bundles - #169
Merged
Conversation
verify_frontend_bundle checks a published cross_dataset.frontend_bundle.v1 directory the way the web reader does: schema version, safe partition paths, recorded SHA-256 for every listed partition, page sequence and count reconciliation, and that each partition carries the manifest's run and snapshot IDs. frontend_bundle_partitions exposes the descriptors in a stable order so publishers can share the same attested file list. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Optional dependency group for uploading evaluation bundles to the Hugging Face dataset; the default install and test suite do not need it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One reproducible command publishes a verified Cross-dataset frontend bundle to the policyengine/microcosm-evaluation dataset in the layout of the live Belgium bundle: <cc>/<run_id>/frontend/... plus <cc>/latest.json. The script verifies the bundle locally (schema, jurisdiction, partition hashes), compares it with the Hub by size and git blob id or LFS SHA-256 so a re-run is a no-op, refuses to overwrite a differing file under an existing run id, uploads the missing files and the latest.json pointer in one commit, then re-downloads manifest.json and summary.json through the public resolve URL and prints the CROSS_DATASET_ARTIFACT_BASE_URL variable the dashboard needs. --dry-run does the verification and prints the plan without network calls; the token comes only from HF_TOKEN or HUGGINGFACE_TOKEN. huggingface_hub is imported lazily so the script and its tests (fake Hub client, no network) run without the publish extra. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Lane bookkeeping, not repository content. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Publishing a Cross-dataset frontend bundle to
policyengine/microcosm-evaluationwas an ad-hoc upload for Belgium (#164). This makes it one verified, idempotent command — ready for the US bundle the moment the #159 run artifacts surface (asked on #166).What changed
scripts/publish_evaluation_bundle_to_hf.py:uv run --extra publish python scripts/publish_evaluation_bundle_to_hf.py --bundle <run>/frontend --jurisdiction US [--repo …] [--dry-run] [--no-latest].cross_dataset.frontend_bundle.v1), the manifest'sjurisdictionscontains the requested one, and a recomputed SHA-256 for every manifest-attested partition. Files the manifest does not attest are reported and never uploaded.<cc>/<run_id>/is refused before any commit (identity = size + LFS sha256, else git blob id).<cc>/latest.jsonland in one commit (parent_commitguards concurrent writers), so the pointer never references a partial run; the previous pointer target is echoed when it moves.CROSS_DATASET_ARTIFACT_BASE_URLfor US,_<CC>for others, matching Cross-dataset page per jurisdiction (BE bundle live; fail closed on manifest jurisdictions) #167;GBaliases touk).--dry-runneeds neither a token norhuggingface_hub; the Hub client is imported lazily, so the default install stays dependency-free (publishoptional group added).evaluation_harness/frontend_bundle.py: the verification is now a reusableverify_frontend_bundle/frontend_bundle_partitions(safe relative paths, page sequence/count reconciliation, per-partition run/snapshot identity) shared by the script.docs/cross-dataset-api.mdgains the publish section;docs/chronicle-update-workflow.mdlinks it as the publish step.Validation
uv run pytest -q: 434 passed (403 baseline + 31 new; fake Hub client only, no network in tests). Without thepublishextra: 433 passed, 1 skipped.--dry-runagainst the real Belgium bundle: 12 files verified, exit 0; jurisdiction mismatch and missing-token paths exit 1 before any network.latest.jsonbyte-matches the live pointer (pinned by a test); a locally edited copy is refused as immutable.🤖 Generated with Claude Code