Skip to content

fix(asvs): refuse an abbreviated anchor SHA — checkout reads a short ref as a branch name - #134

Merged
wshallwshall merged 12 commits into
mainfrom
claude/asvs-full-sha-anchor
Aug 2, 2026
Merged

fix(asvs): refuse an abbreviated anchor SHA — checkout reads a short ref as a branch name#134
wshallwshall merged 12 commits into
mainfrom
claude/asvs-full-sha-anchor

Conversation

@wshallwshall

Copy link
Copy Markdown
Collaborator

actions/checkout resolves ref: as a branch or tag name unless it is the full
40-character hash. An abbreviated anchor_commit therefore died with

A branch or tag with the name '8f01cef8' could not be found

emitted only after it had run git branch --list --remote origin/8f01cef8 and
git tag --list 8f01cef8 — neither of which is what the field means.

check_pinning now refuses anything that is not [0-9a-f]{40}, so it is caught at
authoring time instead of in CI. Two tests; the negative one was proved red first.

Why this is worth a PR rather than a one-line data fix

Fourth instance of one defect in this tool in a day, and all four are the same
statement — the gate failed for a reason having nothing to do with the property it
claims to measure
:

drift gate compared against a CI-injected SHA → could never pass
engine checkout bare SHA without full history → could never resolve
anchor commit pinned to a branch-only commit → died when the branch went
anchor commit abbreviated → read as a branch name

I wrote the invariant that names this class today, in this tool's own docstrings, and
then shipped four violations of it. The discipline is not enforceable by attention —
every one was caught by CI going red. Refusing the short SHA moves this one from
caught late to cannot be written.

Verification

ruff check · ruff format --check · mypy --strict · 26/26 tests pass.
Consumed by the vault gate, which is green on the paired change.

… ref as a branch name

actions/checkout resolves ref: as a BRANCH OR TAG name unless it is the full
40-character hash. An abbreviated anchor_commit therefore dies with
A branch or tag with the name 8f01cef could not be found -- emitted after it ran
git branch --list --remote origin/8f01cef8 and git tag --list 8f01cef, neither of
which is what the field means.

check_pinning now refuses anything that is not [0-9a-f]{40}, so this is caught at
authoring time instead of in CI. Two tests, the negative one proved red first.

FOURTH instance of one defect in this tool in a day, and all four are the same
statement: the gate failed for a reason having nothing to do with the property it
claims to measure.

  drift gate      compared a CI-injected SHA      -> could never pass
  engine checkout bare SHA, no full history       -> could never resolve
  anchor commit   pinned to a branch-only commit  -> died when the branch went
  anchor commit   abbreviated                     -> read as a branch name

Worth recording rather than quietly fixing: I wrote the invariant that names this
class today, in this tools own docstrings, and then shipped four violations of it.
The discipline is not enforceable by attention -- every one was caught by CI going
red. Refusing the short SHA moves this one from caught-late to cannot-be-written.
… that it is quiet

The V5 sweep authored every absence claim in the chapter as a prose narration of a
shell command --

  pattern = "rg -n 'tar.extractall' messagefoundry/ -> exit 1 (zero hits)"

where the field wanted `tar\.extractall\(`. The field's type is `str`; prose is a
valid `str`. The shape permitted it, and only a detector (the BLIND check) caught
it -- nine claims, one chapter, all of which would have shipped proving nothing.

`mutation` closes it: the realistic REINTRODUCTION the pattern claims to exclude,
and the pattern must actually match it. Prose cannot fire on its own stated
reintroduction, so a prose pattern stops being detectable-and-caught and becomes
unwritable. INERT is checked BEFORE the corpus, because a live positive control
must not launder a pattern that could never have matched anything.

BLIND and INERT are different failures and the messages say so: BLIND means the
search could not have SEEN the thing; INERT means the pattern could not have
MATCHED it.

THE TRAP INSIDE THIS FIX, recorded because it is more dangerous than the bug: the
obvious convenience is to derive `mutation` from `pattern` -- unescape it, strip
the anchors, default it in. That makes the check VACUOUS by construction. A value
generated from the thing it validates always satisfies it, so it would pass
forever, on every claim, including the prose ones, while looking like the
strongest check in the file. That is the same defect class this closes, arriving
through the fix. Hence no default, a load-time refusal that says author it, and
a docstring that says why.

The cost, accepted rather than dodged: 47 absence claims already exist without
mutations, and this grandfathers none of them. They must be authored -- not
generated -- before the anchor moves to a commit carrying this code. A required
check that exempts everything already written is the `partial` verdict I would
give anyone else.

Both new tests were proved red first by neutering ONLY the new guard, leaving the
rest of the module intact, so they fail for the reason claimed rather than because
the file stopped importing.
@wshallwshall

Copy link
Copy Markdown
Collaborator Author

Second commit added to this branch (689b3762), which the title no longer covers — flagging rather than letting it ride in quietly.

fix(asvs): an absence claim must prove its pattern CAN fire, not just that it is quiet

The V5 sweep authored every absence claim in the chapter as a prose narration of a shell command — pattern = "rg -n 'tar.extractall' messagefoundry/ -> exit 1 (zero hits)" — where the field wanted tar\.extractall\(. The field's type is str; prose is a valid str. The shape permitted it and only a detector caught it.

mutation is now required: the realistic reintroduction the pattern claims to exclude, which the pattern must actually match. Prose cannot fire on its own stated reintroduction, so it becomes unwritable rather than merely detectable. Checked before the corpus, so a live positive control cannot launder a pattern that could never have matched.

Worth knowing before merge — this is not yet live, by design. The vault gate checks out the engine at anchor_commit and runs scorecard.py from that checkout, so the new check does not take effect until the anchor moves. That is deliberate: 47 existing absence claims have no mutation, and this grandfathers none of them. They have to be authored — not generated — before the anchor advances past this commit. I'll do that backfill after the V8/V14 sweep supplies its own.

The reason for no default is in the docstring and worth repeating: deriving mutation from pattern would make the check vacuous by construction — a value generated from the thing it validates always satisfies it, so it would pass forever on every claim including the prose ones, while looking like the strongest check in the file. Same defect class it closes, arriving through the fix.

Both new tests proved red first by neutering only the new guard, so they fail for the stated reason rather than because the module stopped importing.

@wshallwshall
wshallwshall enabled auto-merge (squash) August 2, 2026 01:08
@wshallwshall
wshallwshall disabled auto-merge August 2, 2026 01:28
… to search

This module lives INSIDE the corpus that absence patterns are searched over
(scripts/ is one of the four scanned package dirs). The guidance string I added
alongside the new `mutation` field carried a literal archive-extraction call as an
example -- so the example itself became a real corpus hit.

Two live absence claims (5.2.5, 5.3.3) flipped to FALSE the moment they were
backfilled: their patterns exclude archive extraction, and the thing they matched
was this file's own advice about how to write them. The engine is unaffected and
still archive-safe; the defect was entirely in the checker's prose.

THIRD instance today of a remedy carrying the disease, and the most literal of the
three:
  - a drift-gate fix that pinned the tree, so the gate could no longer see drift
  - a mutation derived from its own pattern, vacuous by construction (caught first)
  - the DOCUMENTATION of a check breaking the check

Found by the backfill run, which reported both failures rather than tuning the two
claims to pass -- had it 'fixed' them by narrowing the patterns, the contamination
would have survived and two real archive claims would have been silently weakened.

The example now lives only in the Absence docstring, in escaped-regex form, which
cannot self-match. The regression test is an INSTANCE lock, not a class lock, and
says so: it pins the literal that actually bit. The general rule -- write examples
as escaped regex -- is not mechanically enforceable.

The test was proved red twice. The first attempt was a BAD red: reintroducing the
literal inside a string broke the file syntactically, so the test failed on a
collection error rather than on the assertion. Re-done as a comment, which
contaminates the corpus identically while leaving the module parseable -- the test
then failed on its own assertion, which is the only red worth anything.
@wshallwshall
wshallwshall enabled auto-merge (squash) August 2, 2026 01:35
@wshallwshall
wshallwshall disabled auto-merge August 2, 2026 01:45
… to be unique in its file

MEASURED FIRST, because I had just asserted this was unmeasurable. 46 of 292
anchors (15%) cite a token that occurs more than once in its file:

  occurrences: {1: 246, 2: 20, 3: 11, 4: 5, 5: 2, 7: 3, 10: 1, 11: 1, 21: 1, 89: 1, 101: 1}

The tail is not 'at risk'. `await conn.rollback()` occurs 101 times in sqlserver.py,
so ANY line number in that file lands within +/-40 of some occurrence: the anchor
CANNOT fail and certifies nothing. It has been standing as evidence for 2.3.3 for
weeks, green throughout. Same for the 89x sibling. Those two were not anchors
degrading toward hollow -- they were already hollow and reporting success.

This also closes a defect in the REPAIR path rather than the detection path, which
is a shape the taxonomy did not have. Everything else catalogued today fails by
staying silent; this one fails by speaking CORRECTLY and then handing you a repair
it cannot validate. After ADR 0154 landed, 11 anchors went stale and the gate
reported all 11 -- correct. But re-anchoring to the nearest occurrence would have
silently mis-anchored two:

  'UPDATE sessions SET revoked_at=' now has 2 occurrences 19 lines apart -- one the
  keep-N revoke, one a DIFFERENT method ('sign out everywhere else'). Each falls
  inside the other's window, so the check accepts either.

A wrong repair is strictly harder to catch than the fault it replaces:
stale-and-broken is re-detected on the next run, stale-but-resolving never is. And
the repair is exactly where suspicion lapses, because the tool has just proved it
works.

Uniqueness makes the ambiguity UNREPRESENTABLE rather than undetectable: with one
occurrence there is no wrong one to pick, and the 101x anchor is refused at
authoring time instead of certifying a cell indefinitely.

Both tests proved against a neutered guard -- the negative reds, and the positive
still passes, so the check is not simply rejecting everything. That second half
matters here: a guard that rejects all anchors would also make the negative test
green.

No grandfathering. The 46 need longer tokens and I am fixing all 46 before this
merges, same reason as the 47-claim mutation backfill.

Recorded because it is the reason this exists: I told a peer session the hollow
fraction was 'unobservable from inside the system' and could not be measured
without redoing the work. That was a claim generalised one step past the evidence,
made in a message about claims generalised one step past the evidence. The
measurement is fifteen lines. 'Unmeasurable' ends inquiry the way a green light
does, and is more comfortable because it sounds like rigour about limits.
… the gate

14 of 59 decided cells carry ZERO anchors and ZERO absence claims. Nothing about
them is verified, and the gate could not say so, because check_anchors iterates the
evidence a cell HAS. A cell with none is skipped and fails nothing: the check could
only ever validate evidence that existed, never assert that it must exist.

  1.2.2  3.7.3  4.2.1  4.4.1  6.3.3  7.1.3  7.5.2
  11.3.3  11.7.1  12.2.2  13.3.2  16.3.2  16.4.2  (+1)

Most are inherited from the prose lineage -- verdicts that were REACHED but never
ANCHORED -- and ten of them are the same cells the 2026-07-31 reconciliation
recovered after they were dropped from an enumeration. They came back with verdicts
attached and no evidence, which is the exact conflation ADR 0156 exists to prevent:
a guess wearing a verdict's clothes. docs/ASVS-ASSESSMENT-METHOD.md section 3 says
'every non-unverified cell carries at least one anchor'. It was policy, and nothing
enforced it.

FOURTH 'cannot observe itself' defect in this tool, and the only one about absence
of evidence rather than quality of it. The other three were the frozen checkout,
the paths filter that excluded the workflow, and the trigger set that excluded the
subject. This one is subtler: it is not that the check was pointed at the wrong
thing, it is that the check's DOMAIN was the set of things that already existed.
A rule quantified over what is present cannot notice what is missing.

An absence claim counts as evidence: a `fail` is often proved by absence rather
than presence, so the rule is >=1 anchor OR >=1 absence claim. Both halves tested.

Two existing fixtures went red when this landed, correctly -- they carried decided
verdicts with no evidence. Fixed the fixtures rather than the rule, and said so in
the helper's docstring so the next person does not read the placeholder anchor as
cargo.

MEASURED CORRECTION to something I told a peer: they suggested the hollow-anchor
problem meant cell 2.3.3 was 'certified on anchors incapable of failing' and asked
whether the survey percentage was ever real. Measured: NO decided cell rests
entirely on ambiguous anchors (2.3.3 has 4 ambiguous of 8), so the count is not
overstated on that basis. The real hole was this one, which neither of us was
looking at -- worse in kind, since 14 cells have no evidence at all rather than
weak evidence.
@wshallwshall
wshallwshall enabled auto-merge (squash) August 2, 2026 02:49
@wshallwshall
wshallwshall merged commit a05103f into main Aug 2, 2026
32 checks passed
@wshallwshall
wshallwshall deleted the claude/asvs-full-sha-anchor branch August 2, 2026 05:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant