ci: detect a GitHub Action that is not pinned to a commit

[abrichr]

Why

A uses: reference that names a tag resolves to whatever the upstream owner points it at today. The owner can move the tag. Nothing in our repositories records that it moved, so a release can change behaviour with no commit of ours to review, blame, or revert.

Not hypothetical. pypa/gh-action-pypi-publish v1.14.0 bundles twine 6.1.0 and packaging 25.0, which reject Metadata-Version: 2.5 — the version current hatchling emits. It broke the openadapt-evals 0.91.0 release after the tag, the version commit and the GitHub release had landed, so PyPI stayed stale while every other artifact said the version shipped. Twelve repositories were corrected by hand in 2026-08.

Nothing detects the next one. dependency-review.yml reads dependency manifests, not workflow uses: lines. codeql.yml analyses code. Dependabot proposes version bumps but is equally happy to leave a floating tag floating.

What the sweep found

Measured across the 38 non-archived, non-fork repositories we own:

uses: references	count
SHA-pinned	451 (80.1%)
not SHA-pinned	112, across 19 repositories

Two track a branch — superfly/flyctl-actions/setup-flyctl@master — which follows another project's HEAD continuously.

Why a baseline instead of a weekly count

Alerting on 112 items every week is exactly the failure sweep_default_branch_ci.py already warns about: "a daily issue that cries wolf gets muted, and a muted alert is worse than none."

So the accepted backlog lives in action-pin-baseline.json (93 decisions covering those 112 references) and the sweep alerts only on a reference that is new, or worse than what was accepted. The backlog is counted in the report and never raises the alarm, so the number falls because someone chose to work on it — not because a bot nagged.

Validation never regenerates the baseline. --write-baseline does, deliberately, for review — the same shape as public-artifacts.json in openadapt-flow.

Why here, and why weekly

repos.yml is the docs-site registry and omits internal tooling, so a sweep built on it would have missed openadapt-consilium and openadapt-viewer — both of which needed this exact fix in 2026-08. This uses the live organisation listing, the same choice and the same reason as sweep_default_branch_ci.py. Archived repos and forks are excluded.

Weekly, not daily: pin drift is slow and operator-driven, and a daily run would repeat the same answer six extra times a week. 08:29 Monday is clear of every other cron here.

Verification

Run against the live organisation, not just fixtures:

• Quiet with the baseline — 38 repositories, 112 unpinned, 0 needing attention, alert=false clear=true.
• Fires with one row removed — alert=true, naming openadapt-wright deploy.yml:92 and verify-fly-credential.yml:38 as branch-tracking.
• This repository's own workflows: 0 unpinned. The new workflow pins all three of its actions by SHA.
• 18 offline tests, including the exact reference that stranded openadapt-evals 0.91.0, a short SHA, an uppercase SHA, and comment lines that must not false-positive.

One test caught a real bug during development: the same (repo, workflow, ref) appears in several jobs of one file, so the first baseline held 112 rows for 93 decisions while load_baseline silently collapsed them — the committed file disagreed with the file the validator used. Fixed in worst_tier_per_identity.

Cost

One ubuntu-latest runner, standard library only, no dependency install for the sweep itself. Roughly 40 repository reads plus one file read per workflow file, against a 1000/hour budget. This repository is public, so the minutes are free.

🤖 Generated with Claude Code