docs: give the verify command that actually works - #141
Merged
Conversation
Caught by running it against the real 0.5.0-rc.2 release, which is the only place it could be caught: nothing in the suite runs `gh`. The line both READMEs handed users was `gh attestation verify <zip> --bundle <bundle>`, and it fails twice over. `gh` refuses without `--repo` or `--owner`. Past that, it looks for a build-provenance attestation and reports "no attestations found with predicate type" - because the bundle published beside the archive is the SBOM attestation, made after signing over the bytes a user downloads. Both are the tool being precise. The documentation was wrong. Verified in both directions on the released files: with the two flags added it exits 0 and prints nothing, and appending one byte to the archive makes it exit 1. A green that cannot go red would have proved nothing. New guard pins the two halves together: the predicate type the READMEs tell people to ask for must be the one attest-release.yml actually produces, so changing what the workflow attests can no longer break the command silently for every user. A mutation drops the flag from the README and is caught. Also drops a claim I could not support. The old text said the bundle works "with no network at all"; that was never measured, and what is true is narrower - the evidence travels with the download instead of being fetched from the API. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`test_no_stale_app_flags_in_readmes` reads every backticked `--flag` in the READMEs and asserts our own parser has it - which is the right check, and it cannot tell whose flag it is looking at. The verification section now spells out the exact `gh` command a user runs against a release, so `--repo` and `--predicate-type` read as flags this program has lost. They go on the same list that already carries PyInstaller's, with a note saying whose each one is. The list stays short on purpose: its job is to stop a real typo in OUR flags hiding behind somebody else's. Caught by CI on the ubuntu leg, and reproduced locally afterwards in the two guards I had not run - the README ones I did run were green, and this reads the READMEs from a different file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Caught by running it against the real
v0.5.0-rc.2release - the only place it could becaught, because nothing in the suite runs
gh.What was wrong
Both READMEs handed users
gh attestation verify <zip> --bundle <bundle>. It fails twice:ghrefuses without--repoor--owner;found with predicate type" - because the bundle published beside the archive is the
SBOM attestation, made after signing over the bytes a user downloads.
Both are the tool being precise. The documentation was wrong.
Verified in both directions
On the released files: with
--repoand--predicate-type https://spdx.dev/Document/v2.3it exits 0 and prints nothing. Append one byte to the archive and it exits 1
("verifying with issuer sigstore.dev"). A green that cannot go red would have proved
nothing.
New guard
The predicate type the READMEs tell people to ask for must be the one
attest-release.ymlactually produces. Change what the workflow attests and the commandwould otherwise break silently for every user, with nothing in CI able to notice. A
mutation drops the flag from the README and is caught.
Also
Drops a claim I could not support: the old text said the bundle works "with no network at
all". That was never measured. What is true is narrower - the evidence travels with the
download instead of being fetched from the API.
🤖 Generated with Claude Code