Skip to content

feat(preallow): redact credentials in output instead of withholding it, and stop taking a commit hash near "sourcegraph" for a token - #19

Merged
liby merged 2 commits into
mainfrom
feature/preallow-redact-credentials
Oct 29, 2026
Merged

liby merged 2 commits into
mainfrom
feature/preallow-redact-credentials

Conversation

@liby

@liby liby commented Oct 29, 2026

Copy link
Copy Markdown
Member

preallow 0.0.10 redacts what its output scan finds instead of withholding the whole output. Each credential-shaped value, and every other copy of it in the result, property names included, becomes [redacted by preallow: <kind>], and a note beside the result tells the agent that the call ran, that a marker is not a usable value, and that the same call run again goes to the classifier. A failed call's error text is redacted the same way. Before, one token in a long log or a process listing cost the agent the rest of it. The trade-off was the user's choice: a credential the patterns miss, sitting beside one they find, now reaches the model, and the README lists it among the known gaps.

The whole output is still withheld where redacting cannot cover what the model reads: a hit in the file a large result is saved to, which the agent can read later; a hit, or a copy of a value found anywhere else, in a context entry another hook attached, which core keeps whole; and a value the rendered text shows that the tool's record does not hold. The redacted answer is a new { result } without ref, since core reuses the messages a ref names. A redacted call stays allowed for the session, so if the engine refuses the redacted answer, .catch withholds the output.

A redaction has to cover the whole credential, so the spans changed. A PEM key runs to its end marker whatever starts its lines, such as Read's line numbers, or else through its base64 lines; PuTTY keys take their continuation lines and an age identity ends at its fixed length. findLeaks returns each gitleaks finding with its secret group's span, and jwt-base64 and microsoft-teams-webhook, whose groups hold only part of the token, are generated without capturing groups. The plugin's own shapes name the value, so TOKEN= stays readable.

The scan now also reads the tool's record, because Read's line numbers or JSON escaping can hide a shape, such as a netrc password, from the text. A record value counts only where the text the model reads shows it, so the file an Edit result keeps is left alone. Reading records exposed a false positive the line numbers had hidden: an empty API_KEY= took the next line's name for its value. A value now starts on its name's line unless it is quoted.

Separately, gitleaks' sourcegraph-access-token rule took any 40-character hex string in a text naming Sourcegraph, so a commit hash in gh pr view or a git log beside the word withheld the output. The generator drops that alternative and keyword, keeping the sgp_ forms; a legacy token is still found under a name such as SRC_ACCESS_TOKEN= by the plugin's own shapes, and the generator fails if a gitleaks update changes the alternative.

Over 230,286 tool outputs from local transcripts, kept outside this repository, the new scan redacts 492 and withholds 85; every output 0.0.9 withheld by its text is now redacted, apart from the Sourcegraph and empty-assignment false positives it no longer flags, and the redacted outputs keep 83.1% of their characters. Of the 85 withheld, 16 have a hit in the saved file and 69 have a saved file the evaluation could no longer read. No redacted result still holds a match. An output of a megabyte holding 38,000 distinct values takes about 8.5 of the hook's 10 seconds, as it did before this change; an output past the limit is withheld.

liby added 2 commits October 29, 2026 20:03
gitleaks' `sourcegraph-access-token` rule also takes any 40-character
hex string in a text that names Sourcegraph, so every git commit hash
printed beside the word withheld the output; it happened on real
sessions with `gh pr view` and git logs. The generator now takes that
alternative and the `sourcegraph` keyword out of the rule, leaving the
`sgp_` forms, as Betterleaks' rule does. A legacy token without the
prefix differs from a hash only by the name before it, such as
`SRC_ACCESS_TOKEN=`, which the plugin's own secret-assignment shapes
already find. The generator fails if a gitleaks update changes the
alternative, instead of dropping the keyword with the regex unchanged.

This is the first change after 0.0.9 reached users, so it opens 0.0.10
and dates 0.0.9.
An output holding anything credential-shaped was withheld whole, so one
token in a long log or a process listing cost the agent the rest of
it. The plugin now replaces each value it finds, and every other copy
of that value in the result, property names included, with
`[redacted by preallow: <kind>]`, and adds a note beside the result
telling the agent that the call ran, that a marker is not a usable
value, to refer to a credential by where it is kept, and that the same
call run again goes to the classifier. A failed call's error text is
redacted the same way. The user chose this knowing that a credential
the patterns miss, beside one they find, now reaches the model; the
README lists it among the known gaps.

The answer is a new `{ result }` without `ref`, since core reuses the
messages a `ref` names. Where the plugin cannot redact, it still
withholds the whole output: a hit in the file a large result is saved
to, which the agent can read later; a hit, or a copy of a value the
text shows, in a context entry a hook below attached, which core keeps
whole; and a value the rendered text shows that the tool's record does
not hold, since core renders the text from the record. A redacted call
stays listed for the session, so `.catch` withholds the output if the
engine refuses the redacted answer; its reason now says "scanning or
redacting it failed".

The scan also reads the record's strings, since line numbers in a Read
result or JSON escapes hide shapes such as a netrc password from the
text; a record value counts only where core's text shows it, as the
model reads nothing else, so a file an Edit result keeps is left alone.
Reading records surfaced a false positive the line numbers had hidden:
an empty `API_KEY=` took the name on the next line for its value, so a
value now starts on its name's line unless it is quoted.

A redacted value must cover the whole credential. A PEM key runs to its
end marker, whatever starts its lines, or else through its base64
lines; an age identity ends where its fixed length does. `findLeaks`
returns each gitleaks finding with the span of its secret group, and
`jwt-base64` and `microsoft-teams-webhook`, whose groups hold only a
fragment of the token, are generated without capturing groups. The
plugin's own shapes name the value they redact, so a name such as
`TOKEN=` stays readable.

Over a local corpus of transcripts, every output 0.0.9 withheld by its
text, apart from the Sourcegraph false positives and the empty
assignments, is now redacted instead, keeping about four fifths of its
characters, and no redacted result still holds a match.
@liby
liby merged commit 4828311 into main Oct 29, 2026
4 checks passed
@liby
liby deleted the feature/preallow-redact-credentials branch October 29, 2026 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant