Fix false prompt-injection alarm on PR Tooling Safety Check bypass labels - #20130
Conversation
…bels The threat-detection job is a separate LLM that only sees the workflow description plus the agent's output, not the process steps. When the agent correctly applies AI-Tooling-Check-Bypassed to a non-fork PR, the detector misreads the bypass label as the agent being manipulated into skipping its scan and raises a false prompt-injection alarm, aborting the run's label and memory outputs. Give the detector context via threat-detection.prompt. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
github-actions
Bot
added
the
AI-Tooling-Check-Bypassed
…ck-false-injection
pull_request_target grants a read-only GITHUB_TOKEN whenever the pull request modifies files under .github/workflows/ (a GitHub security restriction). The release-notes summary comment then fails with HTTP 403 and fails the whole check, even though the release-notes gate itself passed. Wrap the comment write in a try/catch that skips on 403 with a notice, so posting the comment stays best-effort and workflow-modifying PRs are no longer blocked by this check. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The release-notes comment failure is a non-required check that fails for every PR modifying files under .github/workflows/: pull_request_target grants a read-only GITHUB_TOKEN in that case, so the comment write returns 403. This cannot be fixed from a PR branch because pull_request_target runs the base-branch (main) workflow, and it is already addressed by the dedicated PR #20136. Keep this PR focused on the labelops threat-detection fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ck-false-injection
…t 403 Rewriting the release-notes check in #20081 reduced the token to pull-requests: read, but posting a NEW PR comment (issues.createComment POST) requires pull-requests: write. Brand-new PRs with no existing bot comment (like #20130 and dependency PRs) therefore fail with 403 'Resource not accessible by integration', while PRs with a pre-existing comment pass via the updateComment PATCH path (allowed by issues: write). Restore pull-requests: write (the fork-head checkout removed in #20081 stays removed, so no untrusted code runs and the comment body is passed via env var) and wrap the comment write in try/catch so the best-effort informational comment can never fail the release-notes gate. Supersedes the abandoned PR #20136. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ck-false-injection
|
Note on the red
This branch already contains the fix (restore
|
…ck-false-injection # Conflicts: # .github/workflows/check_release_notes.yml
✅ No release notes required |
T-Gro
left a comment
T-Gro
left a comment
There was a problem hiding this comment.
🤖 This review was generated by AI (@expert-reviewer agent). Findings may contain inaccuracies — please verify independently.
Reviewed the threat-detection hint change and its compiled .lock.yml. The generated lock file is consistent with the .md source (description + CUSTOM_PROMPT env match, frontmatter_hash regenerated), and CUSTOM_PROMPT is consumed by the standard setup_threat_detection.cjs step. One security-relevant scoping concern noted inline.
Address review feedback: the detector hint previously told the threat detector that applying AI-Tooling-Check-Bypassed to any PR is benign. Bypass-labeling is only the designed path for non-fork PRs; a fork PR carrying a bypass label instead of a diff scan is exactly the injection outcome this detector should catch. Scope the reassurance to the non-fork bypass / fork scanned-clean path and explicitly flag a bypass label on a fork PR as a deviation worth flagging. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ck-false-injection
github-project-automation
Bot
moved this from New
to In Progress
in F# Compiler and Tooling
github-project-automation
Bot
moved this from In Progress
to Done
in F# Compiler and Tooling
…bels (dotnet#20130) * Fix false prompt-injection alarm on PR Tooling Safety Check bypass labels The threat-detection job is a separate LLM that only sees the workflow description plus the agent's output, not the process steps. When the agent correctly applies AI-Tooling-Check-Bypassed to a non-fork PR, the detector misreads the bypass label as the agent being manipulated into skipping its scan and raises a false prompt-injection alarm, aborting the run's label and memory outputs. Give the detector context via threat-detection.prompt.
2 participants
The PR Tooling Safety Check runs a separate threat-detection LLM that only sees the workflow description plus the agent's output, not the process steps. When the agent correctly applies
AI-Tooling-Check-Bypassedto a non-fork PR, the detector misreads the bypass label as the agent being manipulated into skipping its scan and raises a false prompt-injection alarm, aborting the run's label and memory outputs.threat-detection.promptnow tells the detector that bypass-labeling non-fork PRs is the designed path defined intooling-check-repo-rules.md, not an injection.