Skip to content

fix: secret scan flagged every on-chain proof artifact as a private key - #100

Closed
celesteanglm wants to merge 1 commit into
mainfrom
fix/secret-scan-false-positives
Closed

fix: secret scan flagged every on-chain proof artifact as a private key#100
celesteanglm wants to merge 1 commit into
mainfrom
fix/secret-scan-false-positives

Conversation

@celesteanglm

Copy link
Copy Markdown
Collaborator

What happened

The first backfill dry run (pr_numbers: all, dry_run: true) had the reviewer tell PR #95:

The biggest thing is that the secret scanner found what look like raw private keys in the aiport-verifier and antfleet-pr-audit packages. That's a hard blocker — those need to be removed from the files and maybe the git history before this can move forward.

All six hits were false positives. What they actually were:

Value Actually
0x94fa44b1… EAS attestation UID, inside a public easscan.org URL
0x740e289a… EAS schema UID
0x82e3fd52… Basescan settlement transaction hash
0xdb8b92ee… deliverableHash
0x41707072… hex-encoded ASCII — the word "Approved"

Why

The pattern was /0x[a-fA-F0-9]{64}/. That is the shape of a private key and the shape of a transaction hash, attestation UID, block hash, or content hash — indistinguishable. Those artifacts are the entire point of a showcase submission, so matching on shape alone gives roughly a 100% false-positive rate on exactly the content this repo exists to collect.

Caught only because the backfill was dry-run first. Posting it would have falsely accused a contributor of leaking a key.

Fix

  • Require key-ish wording (private key, privkey, secret key, seed phrase, mnemonic) near the value; stand down when the value is inside a URL or the surrounding text names a public identifier (tx, hash, uid, attestation, schema, deliverable, …).
  • Tighten the PRIVATE_KEY / mnemonic patterns to require a key-shaped value, not any assignment.
  • Tell the model this scan is an unverified heuristic: check the real value before mentioning it, ask the author to confirm rather than assert a leak, and never recommend rewriting git history.

Verification

  • PR Feat/acp sdk v2 to v3 migration #95's real file set → 0 hits (was 6)
  • Still caught: PRIVATE_KEY=0x…, a key introduced as "private key: 0x…", a 12-word mnemonic
  • Stay silent: tx hashes, attestation UIDs, schema UIDs, deliverableHash, URL-embedded identifiers

Merge this before posting any reviews

The backfill has only ever run in dry-run mode, so nothing was published. This should land before dry_run: false.

🤖 Generated with Claude Code

The first backfill dry run had the reviewer tell PR #95 that raw private keys
were committed, calling it a hard blocker and suggesting a git-history rewrite.
All six hits were false positives:

  0x94fa44b1...  EAS attestation UID, inside a public easscan.org URL
  0x740e289a...  EAS schema UID
  0x82e3fd52...  Basescan settlement transaction hash
  0xdb8b92ee...  deliverableHash
  0x41707072...  hex-encoded ASCII, literally the word "Approved"

The pattern was /0x[a-fA-F0-9]{64}/, which is the shape of a private key and
equally the shape of a transaction hash, attestation UID, block hash or content
hash. Those are the entire point of this repo, so matching on shape alone flags
every genuine proof artifact — a ~100% false-positive rate on exactly the
content a showcase submission is expected to contain. Posting that would have
falsely accused a contributor of leaking a key.

- require key-ish wording (private key, privkey, secret key, seed phrase,
  mnemonic) near the value, and stand down when the value sits inside a URL or
  the surrounding text names a public identifier (tx, hash, uid, attestation,
  schema, deliverable, ...)
- tighten the PRIVATE_KEY and mnemonic patterns to require a key-shaped value
  rather than any assignment
- tell the model in the prompt that this scan is an unverified heuristic, to
  check the actual value before mentioning it at all, and if it does raise one,
  to ask the author to confirm rather than assert a leak — and never to
  recommend rewriting history

Verified: PR #95 now yields zero hits; a PRIVATE_KEY assignment, a key
introduced as "private key", and a 12-word mnemonic are all still caught; and
tx hashes, attestation UIDs, schema UIDs, deliverable hashes and URL-embedded
identifiers all stay silent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@celesteanglm

Copy link
Copy Markdown
Collaborator Author

Closed per maintainer decision — not merging this for now.

For the record: with this unmerged, scan_for_secrets still matches any 0x + 64 hex value, so on-chain identifiers (transaction hashes, EAS attestation/schema UIDs, deliverable hashes) keep reading as possible private keys. Verified affected: PR #95 (6 hits, all false) and PR #70 (2 files). Reviews for PRs containing on-chain proof artifacts should be eyeballed before posting. Branch fix/secret-scan-false-positives still holds the fix if it's wanted later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant