Secret rules fire on commented-out placeholders and rate them critical (45 blocking false positives across 73 repos)

[hyperpolymath]

Summary

The secret-detection rules fire on commented-out placeholder text and rate the result
critical. Because critical is what CI gates on, this blocks merges estate-wide on
documentation.

Measured 2026-09-03 across 73 repositories whose Hypatia gate was repaired and is now
genuinely firing: 45 of ~614 real critical findings (7%) are this false positive.
Zero are real credentials — every hit was read and verified.

Evidence

repos	path	what is actually there
39	.envrc	# export API_KEY="..."
1	satellites/a2mliser/.envrc	same templated line
3	echidnabot.example.toml	# token = "ghp_xxxxxxxxxxxxxxxxxxxx", # token = "glpat-xxxxxxxxxxxxxxxxxxxx", # webhook_secret = "your-webhook-secret"
2	echidnabot.toml	identical commented block
1	lith-http/k8s/base/secret.yaml.template	a k8s template whose header reads DO NOT commit actual secrets to git!

Rules involved: Generic API key, Generic secret, GitLab PAT.

Representative full lines, verbatim from the flagged files:

# export API_KEY="..."
# token = "ghp_xxxxxxxxxxxxxxxxxxxx"
# token = "glpat-xxxxxxxxxxxxxxxxxxxx"
# webhook_secret = "your-webhook-secret"

Every one begins with #. Every value is a literal placeholder.

Two independent defects

1. No comment awareness. The rules match the raw line, so a commented example is
   indistinguishable from an assignment. Shell, TOML, YAML and .envrc all use #.
2. No placeholder recognition. ghp_xxxxxxxxxxxxxxxxxxxx is not a plausible token:
   GitHub PATs are ghp_ + 36 base62 characters, and a run of 20 xs has essentially
   zero entropy. "...", your-webhook-secret and <...>-style values are the same class.

Why it matters more than the count suggests

critical is the gate threshold the estate ships at, so each of these blocks a merge.
The signal-to-noise cost is asymmetric: a reviewer who learns that "Secret found" usually
means a comment will stop reading them, which is precisely how a real leak gets waved
through.

Suggested cure (either alone removes all 45)

• Skip matches on lines whose first non-whitespace character is the file type's comment
  marker, or
• Apply an entropy/shape test to the captured value and drop obvious placeholders
  (repeated-character runs, ..., xxx…, your-*, <...>, changeme).

Comment-skipping is the smaller change; the entropy test is the more general one and
also catches uncommented examples in documentation. A middle option: keep detecting them
but emit info rather than critical, so they inform without gating.

Not in scope here

The same sweep produced 296 Descriptile in retired location criticals. Those are the
estate's own migration gate working correctly and are not a Hypatia defect.

Method: annotations pulled from the Hypatia neurosymbolic scan check run on each of the
73 PRs via the GraphQL annotations(first:100) field; each flagged file then read at the
PR head and inspected by hand.