Skip to content

fix(pm): derive dispatch-gates watch hints from the module body, not from comments and self-test fixtures (#8478) - #8511

Merged
hotlong merged 2 commits into
mainfrom
claude/issue-8478-watchhint-precision
Aug 13, 2026
Merged

fix(pm): derive dispatch-gates watch hints from the module body, not from comments and self-test fixtures (#8478)#8511
hotlong merged 2 commits into
mainfrom
claude/issue-8478-watchhint-precision

Conversation

@hotlong

@hotlong hotlong commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Fixes #8478

extractWatchHints scanned a check script's whole source for quoted path-ish spans, so a path a gate merely named — in a header comment (the scan accepts backticks), or as a self-test fixture — was printed in the Local gates for this card block as a path that gate operates on. That MATCHED column is what a dispatch prompt pastes, and its error is one-directional in the expensive direction: a missing lead costs one card one CI round, while a fabricated lead reaches every dispatch prompt whose file surface brushes a fixture path, indistinguishable from a real one.

What changed

Direction 1 and direction 2 from the card, composed; direction 3 was not needed because the boundary turned out to be mechanically detectable.

  • maskComments(source) — comments (line, block, shebang) are blanked before the scan.
  • maskSelfTests(source) — the body of every top-level self-test function is blanked. The anchor is structural, not a comment convention: a function selfTest() { declaration at column 0, optionally export / async, with any name spelling self-test. Measured over the 61 scripts under scripts/ that carry one — 53 write function selfTest() {, 7 write async function selfTest() {, and 4 use compound names (fixtureSelfTest, selfTestReadSeams, prePushIsArmedSelfTest, decisionTableSelfTest) — 61 of 61 at column 0, none an arrow-function const.
  • Both are built on one left-to-right scan rather than a regex, because both questions a regex cannot answer are live in this tree: release-github-releases.mjs carries a URL in a module-body string (a // a naive rule would swallow the line for), and its markdown regex /^(#{1,6})\s(.*)$|^( + backtick + {3,})/gm contains a backtick — a scanner ignorant of regex literals opens a template literal there and never leaves. The self-test's end is found by counting braces at code positions only: check-engine-double-contract.mjs has 24 column-0 } inside its self-test (TS fixtures in template literals), and a "first column-0 brace" boundary would end the mask at the first one and leak every fixture after it.
  • Leading ../ and ./ are stripped from a hint. Dropping the comments left one gate's only surviving literal unable to match anything: check:pm-skill-ratchet reads new URL('../../.claude/skills/pm-dispatch/SKILL.md', import.meta.url), and before this change it reached a SKILL.md card through the copy of that path in its own header prose. The leading segments are the script's depth, not part of what it watches.

Change is confined to extractWatchHints, its new pure helpers, and the self-test.

Measured, not reasoned

Re-derived on the branch base 3208222 (two PRs landed on this file today, so the card's 03b5f81 numbers moved by one each) and after. Coverage-capable hints:

script card @ 03b5f81 base @ 3208222 after
scripts/pm/dispatch-gates.mjs 45 46 4
scripts/check-adr-0087-registration.mjs 33 34 6
scripts/check-empty-changeset.mjs 35 36 3
scripts/pm/check-skill-id-lint.mjs 2 2 2 (the clean control, unchanged)

Across all 66 discoverable gate scripts: 1144 hints -> 473, and no hint gained that any repo path can reach. Scripts with zero hints — the ones that now fall honestly into the "repo-wide / undetermined" bucket instead of matching through prose — went 7 -> 15; on the card's own path the bucket reads 35 families where it read 27.

Every one of the 671 dropped hints was checked against an independent classifier for occurrences outside comment and self-test lines. The 19 flagged were all resolved: module-relative import specifiers ('../eslint.config.mjs', './regen-artifacts.mjs') that normalise to a bare filename and were inert as hints both before and after, and '../..' (the ROOT constant), which names no file.

End-to-end probes, before -> after:

card path before after what went
packages/runtime/src/index.ts 4 2 ADR-0087 gate + check:changeset-gate-self-tests, both via changeset fixtures
skills/demo/SKILL.md 2 0 check-empty-changeset + its self-test gate, both via fixtures
packages/spec/src/data/filter.zod.ts 14 7 seven gates matched on a bare packages/spec in prose; survivors matched on more specific real literals
.claude/agents/os-dev.md 8 6 check:pm-skill-ratchet (reads only pm-dispatch's SKILL.md) and one more
.claude/skills/pm-dispatch/SKILL.md 6 5 ratchet kept, now via its real literal
scripts/pm/dispatch-gates.mjs 1 1 check:pm-dispatch-gates unchanged
.changeset/breaking.md 7 5 ADR-0087's real .changeset hint kept

Self-test

61 -> 87 cases; all 61 existing cases stayed green, no flips to explain.

The new cases pin both directions — a fixture or prose path is not a hint, a module-body path still is — as offline fixtures for the boundary itself, and as live pins against the real scripts for the population the card measured: the ADR-0087 gate keeps .changeset and loses packages/runtime; check-empty-changeset keeps .changeset and loses skills/demo/SKILL.md; the skill ratchet keeps the SKILL.md it counts and loses references/, which it does not count; the skill-id lint keeps both real hints; this tool keeps the workflow directory it reads and loses the spec paths its fixtures name.

Reverse verification, each with its direction predicted first. Removing the self-test mask reddens exactly 7 cases (the 4 boundary fixtures and the 3 live "no longer claims" pins), every comment case staying green. Removing the comment mask reddens 5 (the 3 comment fixtures plus 2 live pins whose false hint has a comment source as well as a fixture one), while a self-test declaration inside a comment anchors nothing stays green — correctly, since maskSelfTests consults the comment flag independently.

That second ablation earned its keep: on the first run it reddened only 1 of the 3 comment cases. The other two named their paths unquoted inside the comment, and the scan only ever reads quoted spans — so they asserted nothing and would have passed forever. Quoted, the ablation reddens all three. The fixture note now says why.

Known residue, stated plainly

  • A gate whose real population is a git diff computed at runtime (check-empty-changeset on a skills-only PR) is genuinely relevant to paths it names nowhere. Its honest home is the "undetermined" bucket, which it cannot reach while it holds other real literals (.changeset). Same shape as the check:i18n entry that CHANGE_KIND_GATES exists for; unchanged by this PR in either direction.
  • A card editing a gate's own script no longer derives that gate — scripts/check-empty-changeset.mjs stops deriving check:changeset-gate-self-tests. Measured, this was mostly accidental before (14 of 66 gate scripts derived any gate; now 7) and the fix belongs in derive, which resolves those file paths already and discards them. Filed as dispatch-gates never derives a gate for a card that edits the gate's OWN check script — the resolved script paths are already in hand and unused #8509 with the numbers; out of this card's stated scope.

Gates

pnpm check:pm-dispatch-gates green (87/87), pnpm check:nul-bytes green (7660 files, plus a wider control-character self-scan of the changed file), npx eslint scripts/pm/dispatch-gates.mjs --no-inline-config clean. Re-derived against the actual diff, the tool names exactly check:pm-dispatch-gates for this card — no family the dispatch prompt missed.

No changeset: a scripts/-only change releases nothing, so route 2 (skip-changeset) applies.


Generated by Claude Code

claude added 2 commits August 13, 2026 16:28
…8478)

`extractWatchHints` scanned a check script's whole source for quoted path-ish
spans, so a path a gate merely NAMED — in a header comment, or as a self-test
fixture — was printed in the MATCHED column as a path the gate operates on.
That column is what a dispatch prompt pastes, and its error is one-directional
in the expensive direction: a missing lead costs one card one CI round, a
fabricated one is pasted into every prompt whose file surface brushes a fixture.

The scan now reads the module body: comments are masked (direction 2) and the
body of every top-level self-test function is masked (direction 1), the two
composed. The self-test boundary is structural — a `function selfTest() {`
declaration at column 0, the spelling 61 of 61 scripts under `scripts/` use —
never a comment convention, and its end is found by counting braces at code
positions so a `}` inside a fixture string cannot close it early.

Measured across all 66 discoverable gate scripts: 1144 hints -> 473, none
gained that any repo path can reach.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018WuTtyckQa1VcXwgd52JpN
…il (#8478)

Reverse-verifying the comment half — removing the mask and watching the
diagnostics — showed two of the new cases staying GREEN through the ablation.
The scan only ever reads QUOTED spans, and those two fixtures named their paths
bare inside the comment, so they asserted nothing either way. Quoted, the same
ablation reddens all three comment cases.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018WuTtyckQa1VcXwgd52JpN
@vercel

vercel Bot commented Aug 13, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 13, 2026 4:34pm

Request Review

@hotlong hotlong added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Aug 13, 2026 — with Claude
@hotlong
hotlong marked this pull request as ready for review August 13, 2026 17:01
@hotlong
hotlong added this pull request to the merge queue Aug 13, 2026
Merged via the queue into main with commit a6231c7 Aug 13, 2026
24 checks passed
@hotlong
hotlong deleted the claude/issue-8478-watchhint-precision branch August 13, 2026 17:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[finding] dispatch-gates watch hints are read from self-test fixtures and comments, so gates are printed as MATCHED for paths they never read

2 participants