Repository navigation
feat(preallow): redact credentials in output instead of withholding it, and stop taking a commit hash near "sourcegraph" for a token - #19
Merged
Conversation
gitleaks' `sourcegraph-access-token` rule also takes any 40-character hex string in a text that names Sourcegraph, so every git commit hash printed beside the word withheld the output; it happened on real sessions with `gh pr view` and git logs. The generator now takes that alternative and the `sourcegraph` keyword out of the rule, leaving the `sgp_` forms, as Betterleaks' rule does. A legacy token without the prefix differs from a hash only by the name before it, such as `SRC_ACCESS_TOKEN=`, which the plugin's own secret-assignment shapes already find. The generator fails if a gitleaks update changes the alternative, instead of dropping the keyword with the regex unchanged. This is the first change after 0.0.9 reached users, so it opens 0.0.10 and dates 0.0.9.
An output holding anything credential-shaped was withheld whole, so one
token in a long log or a process listing cost the agent the rest of
it. The plugin now replaces each value it finds, and every other copy
of that value in the result, property names included, with
`[redacted by preallow: <kind>]`, and adds a note beside the result
telling the agent that the call ran, that a marker is not a usable
value, to refer to a credential by where it is kept, and that the same
call run again goes to the classifier. A failed call's error text is
redacted the same way. The user chose this knowing that a credential
the patterns miss, beside one they find, now reaches the model; the
README lists it among the known gaps.
The answer is a new `{ result }` without `ref`, since core reuses the
messages a `ref` names. Where the plugin cannot redact, it still
withholds the whole output: a hit in the file a large result is saved
to, which the agent can read later; a hit, or a copy of a value the
text shows, in a context entry a hook below attached, which core keeps
whole; and a value the rendered text shows that the tool's record does
not hold, since core renders the text from the record. A redacted call
stays listed for the session, so `.catch` withholds the output if the
engine refuses the redacted answer; its reason now says "scanning or
redacting it failed".
The scan also reads the record's strings, since line numbers in a Read
result or JSON escapes hide shapes such as a netrc password from the
text; a record value counts only where core's text shows it, as the
model reads nothing else, so a file an Edit result keeps is left alone.
Reading records surfaced a false positive the line numbers had hidden:
an empty `API_KEY=` took the name on the next line for its value, so a
value now starts on its name's line unless it is quoted.
A redacted value must cover the whole credential. A PEM key runs to its
end marker, whatever starts its lines, or else through its base64
lines; an age identity ends where its fixed length does. `findLeaks`
returns each gitleaks finding with the span of its secret group, and
`jwt-base64` and `microsoft-teams-webhook`, whose groups hold only a
fragment of the token, are generated without capturing groups. The
plugin's own shapes name the value they redact, so a name such as
`TOKEN=` stays readable.
Over a local corpus of transcripts, every output 0.0.9 withheld by its
text, apart from the Sourcegraph false positives and the empty
assignments, is now redacted instead, keeping about four fifths of its
characters, and no redacted result still holds a match.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
preallow 0.0.10 redacts what its output scan finds instead of withholding the whole output. Each credential-shaped value, and every other copy of it in the result, property names included, becomes
[redacted by preallow: <kind>], and a note beside the result tells the agent that the call ran, that a marker is not a usable value, and that the same call run again goes to the classifier. A failed call's error text is redacted the same way. Before, one token in a long log or a process listing cost the agent the rest of it. The trade-off was the user's choice: a credential the patterns miss, sitting beside one they find, now reaches the model, and the README lists it among the known gaps.The whole output is still withheld where redacting cannot cover what the model reads: a hit in the file a large result is saved to, which the agent can read later; a hit, or a copy of a value found anywhere else, in a context entry another hook attached, which core keeps whole; and a value the rendered text shows that the tool's record does not hold. The redacted answer is a new
{ result }withoutref, since core reuses the messages arefnames. A redacted call stays allowed for the session, so if the engine refuses the redacted answer,.catchwithholds the output.A redaction has to cover the whole credential, so the spans changed. A PEM key runs to its end marker whatever starts its lines, such as Read's line numbers, or else through its base64 lines; PuTTY keys take their continuation lines and an age identity ends at its fixed length.
findLeaksreturns each gitleaks finding with its secret group's span, andjwt-base64andmicrosoft-teams-webhook, whose groups hold only part of the token, are generated without capturing groups. The plugin's own shapes name the value, soTOKEN=stays readable.The scan now also reads the tool's record, because Read's line numbers or JSON escaping can hide a shape, such as a netrc password, from the text. A record value counts only where the text the model reads shows it, so the file an Edit result keeps is left alone. Reading records exposed a false positive the line numbers had hidden: an empty
API_KEY=took the next line's name for its value. A value now starts on its name's line unless it is quoted.Separately, gitleaks'
sourcegraph-access-tokenrule took any 40-character hex string in a text naming Sourcegraph, so a commit hash ingh pr viewor a git log beside the word withheld the output. The generator drops that alternative and keyword, keeping thesgp_forms; a legacy token is still found under a name such asSRC_ACCESS_TOKEN=by the plugin's own shapes, and the generator fails if a gitleaks update changes the alternative.Over 230,286 tool outputs from local transcripts, kept outside this repository, the new scan redacts 492 and withholds 85; every output 0.0.9 withheld by its text is now redacted, apart from the Sourcegraph and empty-assignment false positives it no longer flags, and the redacted outputs keep 83.1% of their characters. Of the 85 withheld, 16 have a hit in the saved file and 69 have a saved file the evaluation could no longer read. No redacted result still holds a match. An output of a megabyte holding 38,000 distinct values takes about 8.5 of the hook's 10 seconds, as it did before this change; an output past the limit is withheld.