fix(asvs): refuse an abbreviated anchor SHA — checkout reads a short ref as a branch name - #134
Conversation
… ref as a branch name actions/checkout resolves ref: as a BRANCH OR TAG name unless it is the full 40-character hash. An abbreviated anchor_commit therefore dies with A branch or tag with the name 8f01cef could not be found -- emitted after it ran git branch --list --remote origin/8f01cef8 and git tag --list 8f01cef, neither of which is what the field means. check_pinning now refuses anything that is not [0-9a-f]{40}, so this is caught at authoring time instead of in CI. Two tests, the negative one proved red first. FOURTH instance of one defect in this tool in a day, and all four are the same statement: the gate failed for a reason having nothing to do with the property it claims to measure. drift gate compared a CI-injected SHA -> could never pass engine checkout bare SHA, no full history -> could never resolve anchor commit pinned to a branch-only commit -> died when the branch went anchor commit abbreviated -> read as a branch name Worth recording rather than quietly fixing: I wrote the invariant that names this class today, in this tools own docstrings, and then shipped four violations of it. The discipline is not enforceable by attention -- every one was caught by CI going red. Refusing the short SHA moves this one from caught-late to cannot-be-written.
… that it is quiet The V5 sweep authored every absence claim in the chapter as a prose narration of a shell command -- pattern = "rg -n 'tar.extractall' messagefoundry/ -> exit 1 (zero hits)" where the field wanted `tar\.extractall\(`. The field's type is `str`; prose is a valid `str`. The shape permitted it, and only a detector (the BLIND check) caught it -- nine claims, one chapter, all of which would have shipped proving nothing. `mutation` closes it: the realistic REINTRODUCTION the pattern claims to exclude, and the pattern must actually match it. Prose cannot fire on its own stated reintroduction, so a prose pattern stops being detectable-and-caught and becomes unwritable. INERT is checked BEFORE the corpus, because a live positive control must not launder a pattern that could never have matched anything. BLIND and INERT are different failures and the messages say so: BLIND means the search could not have SEEN the thing; INERT means the pattern could not have MATCHED it. THE TRAP INSIDE THIS FIX, recorded because it is more dangerous than the bug: the obvious convenience is to derive `mutation` from `pattern` -- unescape it, strip the anchors, default it in. That makes the check VACUOUS by construction. A value generated from the thing it validates always satisfies it, so it would pass forever, on every claim, including the prose ones, while looking like the strongest check in the file. That is the same defect class this closes, arriving through the fix. Hence no default, a load-time refusal that says author it, and a docstring that says why. The cost, accepted rather than dodged: 47 absence claims already exist without mutations, and this grandfathers none of them. They must be authored -- not generated -- before the anchor moves to a commit carrying this code. A required check that exempts everything already written is the `partial` verdict I would give anyone else. Both new tests were proved red first by neutering ONLY the new guard, leaving the rest of the module intact, so they fail for the reason claimed rather than because the file stopped importing.
|
Second commit added to this branch (
The V5 sweep authored every absence claim in the chapter as a prose narration of a shell command —
Worth knowing before merge — this is not yet live, by design. The vault gate checks out the engine at The reason for no default is in the docstring and worth repeating: deriving Both new tests proved red first by neutering only the new guard, so they fail for the stated reason rather than because the module stopped importing. |
… to search This module lives INSIDE the corpus that absence patterns are searched over (scripts/ is one of the four scanned package dirs). The guidance string I added alongside the new `mutation` field carried a literal archive-extraction call as an example -- so the example itself became a real corpus hit. Two live absence claims (5.2.5, 5.3.3) flipped to FALSE the moment they were backfilled: their patterns exclude archive extraction, and the thing they matched was this file's own advice about how to write them. The engine is unaffected and still archive-safe; the defect was entirely in the checker's prose. THIRD instance today of a remedy carrying the disease, and the most literal of the three: - a drift-gate fix that pinned the tree, so the gate could no longer see drift - a mutation derived from its own pattern, vacuous by construction (caught first) - the DOCUMENTATION of a check breaking the check Found by the backfill run, which reported both failures rather than tuning the two claims to pass -- had it 'fixed' them by narrowing the patterns, the contamination would have survived and two real archive claims would have been silently weakened. The example now lives only in the Absence docstring, in escaped-regex form, which cannot self-match. The regression test is an INSTANCE lock, not a class lock, and says so: it pins the literal that actually bit. The general rule -- write examples as escaped regex -- is not mechanically enforceable. The test was proved red twice. The first attempt was a BAD red: reintroducing the literal inside a string broke the file syntactically, so the test failed on a collection error rather than on the assertion. Re-done as a comment, which contaminates the corpus identically while leaving the module parseable -- the test then failed on its own assertion, which is the only red worth anything.
… to be unique in its file
MEASURED FIRST, because I had just asserted this was unmeasurable. 46 of 292
anchors (15%) cite a token that occurs more than once in its file:
occurrences: {1: 246, 2: 20, 3: 11, 4: 5, 5: 2, 7: 3, 10: 1, 11: 1, 21: 1, 89: 1, 101: 1}
The tail is not 'at risk'. `await conn.rollback()` occurs 101 times in sqlserver.py,
so ANY line number in that file lands within +/-40 of some occurrence: the anchor
CANNOT fail and certifies nothing. It has been standing as evidence for 2.3.3 for
weeks, green throughout. Same for the 89x sibling. Those two were not anchors
degrading toward hollow -- they were already hollow and reporting success.
This also closes a defect in the REPAIR path rather than the detection path, which
is a shape the taxonomy did not have. Everything else catalogued today fails by
staying silent; this one fails by speaking CORRECTLY and then handing you a repair
it cannot validate. After ADR 0154 landed, 11 anchors went stale and the gate
reported all 11 -- correct. But re-anchoring to the nearest occurrence would have
silently mis-anchored two:
'UPDATE sessions SET revoked_at=' now has 2 occurrences 19 lines apart -- one the
keep-N revoke, one a DIFFERENT method ('sign out everywhere else'). Each falls
inside the other's window, so the check accepts either.
A wrong repair is strictly harder to catch than the fault it replaces:
stale-and-broken is re-detected on the next run, stale-but-resolving never is. And
the repair is exactly where suspicion lapses, because the tool has just proved it
works.
Uniqueness makes the ambiguity UNREPRESENTABLE rather than undetectable: with one
occurrence there is no wrong one to pick, and the 101x anchor is refused at
authoring time instead of certifying a cell indefinitely.
Both tests proved against a neutered guard -- the negative reds, and the positive
still passes, so the check is not simply rejecting everything. That second half
matters here: a guard that rejects all anchors would also make the negative test
green.
No grandfathering. The 46 need longer tokens and I am fixing all 46 before this
merges, same reason as the 47-claim mutation backfill.
Recorded because it is the reason this exists: I told a peer session the hollow
fraction was 'unobservable from inside the system' and could not be measured
without redoing the work. That was a claim generalised one step past the evidence,
made in a message about claims generalised one step past the evidence. The
measurement is fifteen lines. 'Unmeasurable' ends inquiry the way a green light
does, and is more comfortable because it sounds like rigour about limits.
… the gate 14 of 59 decided cells carry ZERO anchors and ZERO absence claims. Nothing about them is verified, and the gate could not say so, because check_anchors iterates the evidence a cell HAS. A cell with none is skipped and fails nothing: the check could only ever validate evidence that existed, never assert that it must exist. 1.2.2 3.7.3 4.2.1 4.4.1 6.3.3 7.1.3 7.5.2 11.3.3 11.7.1 12.2.2 13.3.2 16.3.2 16.4.2 (+1) Most are inherited from the prose lineage -- verdicts that were REACHED but never ANCHORED -- and ten of them are the same cells the 2026-07-31 reconciliation recovered after they were dropped from an enumeration. They came back with verdicts attached and no evidence, which is the exact conflation ADR 0156 exists to prevent: a guess wearing a verdict's clothes. docs/ASVS-ASSESSMENT-METHOD.md section 3 says 'every non-unverified cell carries at least one anchor'. It was policy, and nothing enforced it. FOURTH 'cannot observe itself' defect in this tool, and the only one about absence of evidence rather than quality of it. The other three were the frozen checkout, the paths filter that excluded the workflow, and the trigger set that excluded the subject. This one is subtler: it is not that the check was pointed at the wrong thing, it is that the check's DOMAIN was the set of things that already existed. A rule quantified over what is present cannot notice what is missing. An absence claim counts as evidence: a `fail` is often proved by absence rather than presence, so the rule is >=1 anchor OR >=1 absence claim. Both halves tested. Two existing fixtures went red when this landed, correctly -- they carried decided verdicts with no evidence. Fixed the fixtures rather than the rule, and said so in the helper's docstring so the next person does not read the placeholder anchor as cargo. MEASURED CORRECTION to something I told a peer: they suggested the hollow-anchor problem meant cell 2.3.3 was 'certified on anchors incapable of failing' and asked whether the survey percentage was ever real. Measured: NO decided cell rests entirely on ambiguous anchors (2.3.3 has 4 ambiguous of 8), so the count is not overstated on that basis. The real hole was this one, which neither of us was looking at -- worse in kind, since 14 cells have no evidence at all rather than weak evidence.
actions/checkoutresolvesref:as a branch or tag name unless it is the full40-character hash. An abbreviated
anchor_committherefore died withemitted only after it had run
git branch --list --remote origin/8f01cef8andgit tag --list 8f01cef8— neither of which is what the field means.check_pinningnow refuses anything that is not[0-9a-f]{40}, so it is caught atauthoring time instead of in CI. Two tests; the negative one was proved red first.
Why this is worth a PR rather than a one-line data fix
Fourth instance of one defect in this tool in a day, and all four are the same
statement — the gate failed for a reason having nothing to do with the property it
claims to measure:
I wrote the invariant that names this class today, in this tool's own docstrings, and
then shipped four violations of it. The discipline is not enforceable by attention —
every one was caught by CI going red. Refusing the short SHA moves this one from
caught late to cannot be written.
Verification
ruff check·ruff format --check·mypy --strict· 26/26 tests pass.Consumed by the vault gate, which is green on the paired change.