ci: answer the Scorecard report, and pin what the scanner cannot see - #134
Merged
Conversation
The build-provenance attestation shipped with the last change and both READMEs explain how to verify it, but CHANGELOG.md never said so - and a user reads the changelog to find out what is new. Rule: anything visible from outside goes in there, and a new way to verify a release is as visible as it gets. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…flow The top-level block held `contents: write`, `id-token: write` and `attestations: write`, so every job added to this file later would inherit them - in the one workflow that can publish an asset under this project's name. They now sit on the single `release` job, and the file itself grants `contents: read`. ci.yml, pages.yml and scorecard.yml already worked this way; this was the last file breaking the rule. Raised by OpenSSF Scorecard as Token-Permissions, 0 out of 10. The install step also loses `python -m pip install --upgrade pip`, which fetched an unpinned pip from the index and then used it to verify the artefact hashes on the very next line. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
requirements-lint.txt is now installed with --require-hashes, which is why it carries its whole closure rather than three names: ruff, mypy, diff-cover and the ten packages underneath them. pip turns hash-checking on for an entire install as soon as one requirement carries a hash, so every package it resolves has to be written down. The closure was resolved with pip for Python 3.14 and came back identical on manylinux2014_x86_64 and win_amd64, so no line needs an environment marker. semgrep and pip-audit move to a new requirements-scan.txt, pinned by version and deliberately not by hash. Their closures are tens of packages that move on their own schedule, and against that: both run with a read-only token, both produce a report, and neither can put a byte into a release, because the build and release jobs install only from requirements.txt and requirements-build.txt and both of those are hash-checked. semgrep cannot be frozen in the sense that matters anyway - the pin fixes the engine, and the rules are fetched from the registry at scan time. The cost is named in that file rather than hidden: a scanner will keep counting the semgrep job's install as unpinned, and it is right to. The tests job installs the lint file whole instead of grepping diff-cover out of it. A hash cannot travel through a command substitution: grep hands pip a bare pin, and pip stays out of hash-checking mode. With semgrep moved out, installing the file no longer drags a 57 MB engine onto both matrix legs. `python -m pip install --upgrade pip` is gone from the four places left in these two workflows, for the reason given in the previous commit. Verified rather than assumed: the lint file installs into a clean virtual environment under --require-hashes, and the tests job's three install commands were replayed in order, because pytest and the lint closure share pluggy. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Five checks in tests/test_version_and_release.py, and four mutation-registry entries that were all caught on the first run: * the two requirement files, each held to its own promise - the lint file fully hashed (a shared helper now checks it and the runtime file with one piece of code), the scan file pinned with == at least; * no workflow may bootstrap pip from the index again; * every install of a hashed file must pass --require-hashes. pip does turn hash-checking on by itself, so without the flag the day a line loses its hash block is the day the check silently stops being mandatory; * release.yml may not grant a write permission at the file level. The pip-bootstrap guard exists because the scanner is blind to most of what it covers. OpenSSF Scorecard skips workflow steps that run in a Windows shell, and three of the five bootstraps were in windows-latest jobs, so it named two and said nothing about the rest - 8 of the 13 unpinned pip commands in this repository. Fixing exactly what a scanner lists would have produced a perfect score with five unpinned installs still in the tree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
The first OpenSSF Scorecard report raised 16 findings on
master. This closes thefive that are worth closing in the tree, answers four in writing, and leaves seven
to time or to arithmetic. The interesting half is what it does not do.
Fixed
release.ymlgrants write on the job, not on the file (Token-Permissions, high,0/10).
contents: write,id-token: writeandattestations: writemoved onto thesingle
releasejob; the file grantscontents: read.ci.yml,pages.ymlandscorecard.ymlalready worked this way, so this was an inconsistency, not adecision. It is not cosmetic while there is one job: a permission written at the top
is inherited by every job added later, by an author with no reason to scroll up, in
the one workflow that publishes under this project's name.
pip install --upgrade pipis gone from all five places it appeared. It fetchedan unpinned pip from the index and then used it to verify the artefact hashes on the
next line - the one unchecked link in a hash-checked chain. The pip that checks them
is now the one
setup-pythonshipped, and that action is pinned by SHA.requirements-lint.txtis hash-checked and holds its whole closure (13 packages:ruff,mypy,diff-coverand the ten underneath them). Resolved with pip forPython 3.14 and identical on
manylinux2014_x86_64andwin_amd64, which is why noline carries an environment marker.
requirements-scan.txtis new and is the one file here pinned by version and notby bytes:
semgrepandpip-audit. Reasoning is written in the file.The
testsjob installs the lint file whole instead of greppingdiff-coveroutof it. A hash cannot travel through a command substitution - grep hands pip a bare
pin and pip stays out of hash-checking mode.
The measurement behind it
🔴 Scorecard does not read workflow steps that run in a Windows shell. From its
source: "Skip unsupported shells. We don't support Windows shells". Three of the
five
--upgrade piplines were inwindows-latestjobs, as were twopip install -r requirements-lint.txtlines it never mentioned. It listed 8 of the13 unpinned pip commands in this repository.
Fixing exactly what it named would have produced Pinned-Dependencies 10/10 with five
unpinned installs still in the tree. An alert list is a sample, not an inventory -
which is why the guard for this lives in the suite and not in the scanner.
Also read from source rather than assumed:
--require-hashesis the only thingthat makes a pip command count as pinned (a version in the file does not), and the
flag cannot be added for show, because pip refuses the install without hashes behind
it.
Deliberately not done
requirements-dev.txtstays unpinned.pytest,hypothesisandpytest-covtrack latest on purpose - that drift is what the weekly run exists to see. Freezing
them to satisfy a scanner trades a real signal for a number. Two alerts closed as
won't-fix.
computeFinalScorereturns earlywhen a tier is not maxed, so the nine required status checks this repository
already has (Tier 3) are never reached while Tier 2 (at least one approving review)
is unmet - and nobody can approve their own pull request. Raising the approval count
behind an admin bypass would move the number and make the rule theatre. Code-Review
(0/29 approved) is the same wall from the other side.
recognises Python through exactly one pattern,
import atherisin a*.pyfile.Property-based testing counts for six other languages and not for Python, so the
hypothesis suites on the filter mini-language and the CLI are invisible to it.
younger than 90 days, and one pull request out of 29 predating those jobs. They
close themselves.
Guards
Five checks in
tests/test_version_and_release.pyand four mutation-registryentries, all four caught on the first run: the two requirement files each held to
its own promise, no workflow bootstrapping pip from the index, every install of a
hashed file passing
--require-hashes, and no write permission at the file level inrelease.yml.Verification
ruff --select F,B,C90: clean.mypy: clean.--require-hashes.testsjob's three install commands replayed in order, because pytest and thelint closure share
pluggy.packages.
Also in this branch
The first commit is unrelated to the above: a changelog line telling users they can
check where a download came from, which the previous change shipped without
announcing.
🤖 Generated with Claude Code