From 65c796a6c00302e0efccce738b9a1aa56817b315 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Wed, 9 Sep 2026 00:39:24 +0100 Subject: [PATCH] sec(gitleaks): add a gating full-history pass beside the working-tree pass The PR secret gate scanned only the checked-out working tree, so a secret that was committed and later deleted was permanently invisible to it. Owner ruling 2026-09-09: "Make the PR gate full-history now." This adds a SECOND gating pass rather than converting the existing one, because the two passes see different populations. Measured on a fixture with this exact pinned binary (8.18.4), the same secret in two placements: committed then deleted working tree: MISSED history: FOUND untracked file working tree: FOUND history: MISSED git mode scans commit diffs and cannot see a file that was never committed; --no-git walks the checked-out tree and cannot see a blob no longer in it. Dropping either pass REMOVES coverage, so both gate. Three changes, all inside the existing `gitleaks` job: 1. fetch-depth: 0 on the checkout. Without it the history pass would walk a single commit and report a pass. 2. The stale comment that told readers to remove --no-git is replaced with the two-population explanation. 3. A new step, "Gitleaks secret scan - git history (gating)", placed immediately after the existing pass, which is left byte-identical. The new step carries its own vacuity control: it asserts `git rev-parse --is-shallow-repository` is false and refuses to report a pass otherwise, because gitleaks' own "scanned N commits" counter under-reports and is not proof of clone depth. It also asserts `test -s "$CONFIG"`, so a silently-failed baseline staging step cannot leave gitleaks scanning almost nothing and calling it green. CHECK-NAME IDENTITY PRESERVED. Callers have `secret-scan / gitleaks` as a required context; a renamed job would make that context go absent, and an absence is never in a failure set. Verified structurally on the parsed YAML tree, not by eye: job keys identical gitleaks, rust-secrets, shell-secrets job name: fields identical all three nil (unchanged) steps 9 -> 10, exactly one added, zero removed existing gating step parsed-identical before and after only pre-existing delta `with:` on the checkout new step position immediately after the working-tree pass --no-git argument lines 2 (working tree + AsciiDoc mirror), unchanged rust-secrets / shell-secrets entirely unchanged KNOWN GAP, stated in the file rather than implied: the AsciiDoc mirror pass copies files to a scratch directory and scans it with --no-git, so fetch-depth: 0 cannot reach it. An AsciiDoc secret that was committed and later deleted stays invisible. "The gate is full-history" holds for every file class but that one. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01FmGveHYaPLDUZVKGR1rLaK --- .github/workflows/secret-scanner-reusable.yml | 69 +++++++++++++++++-- 1 file changed, 65 insertions(+), 4 deletions(-) diff --git a/.github/workflows/secret-scanner-reusable.yml b/.github/workflows/secret-scanner-reusable.yml index 3a8d73b9..99e493ae 100644 --- a/.github/workflows/secret-scanner-reusable.yml +++ b/.github/workflows/secret-scanner-reusable.yml @@ -112,6 +112,12 @@ jobs: contents: read steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + # Full history. One of the two gating passes below scans git history, + # and the default depth-1 clone would silently reduce it to a single + # commit. That pass asserts non-shallowness for itself rather than + # trusting this line to have taken effect. + fetch-depth: 0 # Gitleaks runs as a pinned, checksum-verified binary rather than via # gitleaks/gitleaks-action. The action injects gitleaks into the runner @@ -210,10 +216,15 @@ jobs: rm -rf .standards-gitleaks test -s .gitleaks-estate.toml - # Scans the checked-out working tree (--no-git): deterministic and fast. - # A finding exits non-zero and fails the check. Full-history scanning is - # available by removing --no-git and adding `fetch-depth: 0` to the - # checkout above, at the cost of a slower scan on every PR. + # TWO gating passes run below and they are not redundant. Each sees a + # population the other cannot: + # * working tree (--no-git) -- sees untracked and generated files; + # cannot see a blob deleted in an earlier commit + # * git history -- sees committed-then-deleted secrets; + # cannot see a file that was never committed + # Both gate: a finding in either exits non-zero and fails the check. + # Full history costs a slower scan on every PR. That is the accepted + # price of a deleted secret no longer being permanently invisible. # ── Estate baseline sharing ────────────────────────────────────────── # A consumer repo can inherit the estate-wide allowlist instead of # copying it, by writing in its own .gitleaks.toml: @@ -310,6 +321,56 @@ jobs: --config "$CONFIG" \ --exit-code 1 + # Full-history pass — ADDITIVE, not a replacement. Measured on a fixture + # with this exact pinned binary, the same secret in two placements: + # + # committed then deleted working tree: MISSED history: FOUND + # untracked file working tree: FOUND history: MISSED + # + # git mode scans commit diffs, so it cannot see a file that was never + # committed; --no-git walks the checked-out tree, so it cannot see a blob + # that is no longer in it. Dropping either pass REMOVES coverage. Keeping + # both is why this was added as a new step rather than by editing the one + # above, whose name and log lines existing triage keys on. + # + # KNOWN GAP, stated rather than implied: the AsciiDoc mirror pass below + # scans a filesystem copy with --no-git, so fetch-depth: 0 cannot reach + # it. An AsciiDoc secret that was committed and later deleted stays + # invisible. "The gate is full-history" holds for every file class but + # that one. + - name: Gitleaks secret scan — git history (gating) + run: | + set -euo pipefail + + # fetch-depth: 0 on the checkout is load-bearing HERE. If it ever + # regresses to the default depth-1 clone, detect would walk a single + # commit, find nothing and report a pass — a gate that cannot fail. + # Assert completeness from git itself: gitleaks' own "scanned N + # commits" line under-reports and is not proof of depth. + if [ "$(git rev-parse --is-shallow-repository)" != "false" ]; then + echo "::error::checkout is shallow -- a history scan here would be vacuous; refusing to report a pass" + exit 1 + fi + echo "history available to this scan: $(git rev-list --count HEAD) commits" + + # Same config resolution as the working-tree pass, duplicated rather + # than factored out so that step stays byte-identical. + CONFIG=".gitleaks-estate.toml" + if [ -f .gitleaks.toml ]; then + CONFIG=".gitleaks.toml" + fi + # A staging step that failed silently would leave gitleaks running a + # reduced config over almost nothing and calling the result green. + test -s "$CONFIG" + + "$RUNNER_TEMP/gitleaks" detect \ + --source . \ + --redact \ + --no-banner \ + --verbose \ + --config "$CONFIG" \ + --exit-code 1 + # AsciiDoc blind-spot pass — MEASURED, not theoretical. # # gitleaks' DEFAULT global allowlist excludes paths matching