Repository navigation
Keep requested reviewbot reviews from reading the pull request id #63
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -529,9 +529,14 @@ spec: | |
| them for it in the thread and end the turn — never guess a repository. | ||
| 2. Resolve the head commit — always name the repository with --repo, which | ||
| is how reviewbot authorizes it (a positional selector is refused): | ||
| gh pr view <number> --repo <owner>/<repo> --json headRefOid | ||
| gh pr view <number> --repo <owner>/<repo> --json headRefOid,author,isDraft | ||
| (or `gh api repos/<owner>/<repo>/pulls/<number>`), then get the code | ||
| and review it exactly as in steps 2-3 above. | ||
| NEVER include `id` in the --json fields in this mode. Reading a pull | ||
| request's node id records that pull request as one this session | ||
| works on, and only a session a webhook opened is bound to its pull | ||
| request — here the read is refused outright with "has no slot-bound | ||
| grant for this session". Nothing in this mode needs the id. | ||
| 3. Build the report (step 4) and DELIVER IT TO THE THREAD (step 5: | ||
| respond_to_user with the report in `attached`, then | ||
| artifact_offer_view). The artifact-delivered requirement still | ||
|
|
@@ -554,8 +559,15 @@ spec: | |
| done once the review is delivered to the thread. | ||
|
|
||
| If a git or gh call is refused because the repository is not in the | ||
| allowlist, that is by design: relay the refusal to the thread and stop — | ||
| do not retry or try another spelling of the URL. | ||
| allowlist — the refusal itself says "not in reviewbot's allowlist" — | ||
| that is by design: relay the refusal to the thread and stop — do not | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [minor] The allowlist "relay and stop" rule is unscoped and conflicts with the new multi-PR rule Problem: this sentence says to relay the refusal "and stop". Lines 568-570 say "A refusal or failure on one does not stop the rest." For a request naming two PRs where the second repo is not allowlisted, the model may end the whole turn instead of continuing to the other PR. The multi-PR paragraph also never says to report each failed PR in the thread. The "If you cannot review" section (line 510) is written around conclude_trigger_status, which this mode does not have, so a clone failure on one PR can end silently. |
||
| retry or try another spelling of the URL. Any OTHER refusal is not an | ||
| allowlist refusal; never describe it as one. A "no slot-bound grant" | ||
| refusal means the call asked for `id` (step 2): drop it and retry. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [minor] "Drop it and retry" has no bound and no outcome for a failed retry Problem: the instruction does not say how many retries are allowed or what to report if the retry is also refused. It sits next to "do not retry" for the allowlist case, so the retry rule is easy to misapply. |
||
|
|
||
| Several pull requests in one request: review them one at a time, each | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [minor] Multi-PR flow does not give each PR its own artifact Problem: "each exactly as above" pulls in step 4 (line 428-429), which says "On a later round pass |
||
| exactly as above, and deliver each review as its own message. A refusal | ||
| or failure on one does not stop the rest. | ||
|
|
||
| ## Pausing and resuming a pull request | ||
| Someone may ask you to stop reviewing a particular pull request — "stop | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[minor] Prompt matches refusal text by hand; nothing records the dependency
Problem: this paragraph keys on the literal phrase "not in reviewbot's allowlist" (examples/reviewbot/manifests/toolspecs.yaml:83 and :189) and on "no slot-bound grant" (pkg/authz/relwrites/relwrites.go:164). Rewording either message silently breaks the discriminator, and no test notices.
Fix: add a YAML comment next to both
message:lines in toolspecs.yaml (83, 189) saying agentclass.yaml's Requested review section matches the phrase "not in reviewbot's allowlist", so the message must keep it. Put the comment in toolspecs.yaml, not inside this block scalar, where it would become prompt text.Test: do not add a test that reads examples/.
Constraints: CLAUDE.md "Tests MUST NOT reference
examples/" rules out a test that pins the prompt text.