-
Notifications
You must be signed in to change notification settings - Fork 879
Consolidate AI review CI quality gate #3550
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
8c8adff
c19330e
6ffc154
582705c
9c516f1
be8358c
65e6a6e
caac279
c61fab4
79f234d
24d09bd
7dbfa5c
8602d56
341b6f5
f638378
2d6455a
c40c07b
f3a7add
cba2503
74dd752
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,131 @@ | ||
| name: AI Review | ||
|
|
||
| on: | ||
| # Seed / refresh the status when the PR is opened, pushed to, or marked ready. | ||
| # This is the trigger we've actually seen fire, so it carries the workflow even | ||
| # if check_run events don't reach it. | ||
| pull_request: | ||
| types: [ opened, synchronize, reopened, ready_for_review ] | ||
| # Re-evaluate the moment a reviewer's check changes (so we don't wait for the | ||
| # next push). Filtered to our reviewers by name in the job-level `if`. | ||
| check_run: | ||
| types: [ created, completed, rerequested ] | ||
| merge_group: | ||
|
|
||
| permissions: | ||
| checks: read | ||
| statuses: write | ||
|
|
||
| jobs: | ||
| evaluate: | ||
| # NOTE: the 'Bugbot' / 'Claude' tokens below (and the keys in the script) must | ||
| # appear in the reviewers' real check-run names. Case-insensitive substring match, | ||
| # so suffixes like "Claude Code Review / review" or casing tweaks are fine. | ||
| if: > | ||
| github.event_name == 'merge_group' || | ||
| (github.event_name == 'pull_request' && github.event.pull_request.draft == false) || | ||
| (github.event_name == 'check_run' && | ||
| (contains(github.event.check_run.name, 'Bugbot') || | ||
| contains(github.event.check_run.name, 'Claude'))) | ||
| # Serialize every run for the same head SHA so the last run wins | ||
| # (commit statuses are last-write-wins). | ||
| concurrency: | ||
| group: >- | ||
| ai-review-${{ github.event.check_run.head_sha | ||
| || github.event.pull_request.head.sha | ||
| || github.event.merge_group.head_sha }} | ||
| cancel-in-progress: false | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 5 | ||
| steps: | ||
| - name: Generate GitHub App token | ||
| id: generate-token | ||
| uses: actions/create-github-app-token@v2 | ||
| with: | ||
| app-id: ${{ secrets.PLATFORM_CODE_AGENT_APP_ID }} | ||
| private-key: ${{ secrets.PLATFORM_CODE_AGENT_APP_PK }} | ||
|
claude[bot] marked this conversation as resolved.
|
||
| - uses: actions/github-script@v9 | ||
| with: | ||
|
cursor[bot] marked this conversation as resolved.
|
||
| github-token: ${{ steps.generate-token.outputs.token }} | ||
| script: | | ||
|
claude[bot] marked this conversation as resolved.
|
||
| // Match each reviewer by a distinctive (case-insensitive) substring of | ||
| // its check-run name, so a suffix/casing tweak still resolves. Tighten a | ||
| // key if it ever collides with an unrelated check. | ||
| const REVIEWERS = [ | ||
| { label: "Cursor Bugbot", key: "bugbot" }, | ||
| { label: "Claude Code Review", key: "claude" }, | ||
| ] | ||
| const CONTEXT = "AI Review" | ||
| const PASSING = ["success"] | ||
| const ADVISORY = ["neutral", "skipped"] // accepted, non-blocking | ||
| const REJECTING = ["failure", "action_required"] // genuine "changes needed" | ||
|
|
||
| // Head SHA the reviewers ran against. For check_run events the workflow | ||
| // file is read from the default branch, but this SHA still points at the | ||
| // PR head, so the status lands in the right place. | ||
| const sha = | ||
| context.payload.check_run?.head_sha ?? | ||
| context.payload.pull_request?.head?.sha ?? | ||
| context.payload.merge_group?.head_sha | ||
|
|
||
| const targetUrl = | ||
| `${context.serverUrl}/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}` | ||
| const setStatus = (state, description) => | ||
| github.rest.repos.createCommitStatus({ | ||
| owner: context.repo.owner, repo: context.repo.repo, sha, | ||
| state, context: CONTEXT, | ||
| description: description.slice(0, 140), target_url: targetUrl, | ||
| }) | ||
|
|
||
| // Merge queue: reviewers don't re-run here, so just pass. | ||
| if (context.eventName === "merge_group") { | ||
| await setStatus("success", "Merge queue — auto-pass") | ||
| return | ||
| } | ||
|
|
||
| // One pass over every check on the commit, then pick the latest per | ||
| // reviewer (most recently started wins). | ||
| const allRuns = await github.paginate(github.rest.checks.listForRef, { | ||
| owner: context.repo.owner, repo: context.repo.repo, ref: sha, per_page: 100, | ||
| }) | ||
| const at = r => new Date(r.started_at ?? r.updated_at ?? r.created_at ?? 0) | ||
| const latestFor = key => | ||
| allRuns.filter(r => r.name.toLowerCase().includes(key)).sort((a, b) => at(b) - at(a))[0] | ||
| const runs = REVIEWERS.map(r => latestFor(r.key)) | ||
|
Comment on lines
+91
to
+94
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 The fallback chain Extended reasoning...What the bug isLine 91 defines the sort key for picking the latest check_run per reviewer: const at = r => new Date(r.started_at ?? r.updated_at ?? r.created_at ?? 0)The fallback chain was added in response to inline comment 3372143715 (and is the same shape the author proposed for the rerun-staleness problem cursor[bot] flagged in 3363611828). But per the GitHub Checks Runs REST API schema, a check_run object only has these top-level fields: Why this matters during rerunsWhen a maintainer clicks Re-run on Cursor Bugbot or Claude Code Review, GitHub creates a fresh check_run with
Why existing code doesn't prevent itThe author clearly intended the fallback chain to handle this case — it was added specifically in response to the queued-started_at-is-null problem cursor[bot] flagged. The fix landed but doesn't work because the chosen fallback fields are not in the check_run schema. This is a near-miss, not a missed-case: the structural intent is correct, the field names are wrong. ImpactOnce
Step-by-step proof (stale success → unsafe merge)
The runtime semantics of How to fix itSeveral viable options, in roughly increasing order of robustness:
const latestFor = key => {
const matches = allRuns.filter(r => r.name.toLowerCase().includes(key))
const inFlight = matches.find(r => r.status !== 'completed')
if (inFlight) return inFlight
return matches.sort((a, b) => new Date(b.started_at ?? 0) - new Date(a.started_at ?? 0))[0]
}
const at = r => [new Date(r.started_at ?? 0).getTime(), r.id]
const latestFor = key => allRuns
.filter(r => r.name.toLowerCase().includes(key))
.sort((a, b) => { const [ta, ia] = at(a), [tb, ib] = at(b); return tb - ta || ib - ia })[0]
Any of these unsticks the rerun-staleness case the original fallback chain was meant to address. |
||
|
|
||
| const blockers = [] // posted a blocking verdict | ||
| const advisory = [] // accepted but non-blocking (neutral/skipped) | ||
| const waiting = [] // not finished, or finished without a usable verdict | ||
| REVIEWERS.forEach(({ label }, i) => { | ||
| const run = runs[i] | ||
| const c = run?.conclusion | ||
| if (!run || run.status !== "completed") { | ||
| waiting.push(`${label} (${run ? "running" : "not started"})`) | ||
| } else if (REJECTING.includes(c)) { | ||
| blockers.push(label) | ||
| } else if (PASSING.includes(c)) { | ||
| // all good | ||
| } else if (ADVISORY.includes(c)) { | ||
| advisory.push(c === "neutral" ? label : `${label} (${c})`) | ||
| } else { | ||
| waiting.push(`${label} (${c ?? "unknown"}; rerun needed)`) | ||
| } | ||
| }) | ||
|
|
||
| // FAILURE: at least one reviewer wants changes. | ||
| if (blockers.length) { | ||
| const verb = blockers.length > 1 ? "have" : "has" | ||
| await setStatus("failure", `Nope. ${blockers.join(", ")} ${verb} blocking notes`) | ||
| return | ||
| } | ||
|
|
||
| // SUCCESS: everyone finished and nothing is blocking. | ||
| if (!waiting.length) { | ||
| await setStatus("success", | ||
| advisory.length ? `Not gonna block it — ${advisory.join(", ")}` : "Good to go — all clear") | ||
| return | ||
| } | ||
|
|
||
| // PENDING: still waiting on a reviewer. | ||
| const done = REVIEWERS.length - waiting.length | ||
| await setStatus("pending", `Review pending (${done}/${REVIEWERS.length} done) — ${waiting.join(", ")}`) | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,7 @@ | ||
| name: ECR | ||
|
|
||
| BREAK AND SEE IF AI CATCHES THIS!!!! | ||
|
Check failure on line 3 in .github/workflows/ecr.yml
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Accidental debug line in workflowHigh Severity A non-comment line Reviewed by Cursor Bugbot for commit 74dd752. Configure here. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 The PR adds the literal line Extended reasoning...What the bug is
How it manifestsRunning a YAML parser against the file fails: GitHub Actions uses the same YAML semantics at workflow load time, so when this lands on Why existing code does not prevent itThe line is a top-level addition with no surrounding guard. The workflow is otherwise correct, but a single malformed line at the document root is enough for GitHub to reject the entire file — there is no partial parsing or "skip bad lines" fallback. Impact
For each push, it builds and pushes three image tags to
Once this PR merges, every push to Why this happened and how to fixThe latest commit on this PR is Fix: delete lines 2–3 of name: ECR
on:
push:
...Step-by-step proof
🔬 also observed by cursor |
||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
|
|
||


Uh oh!
There was an error while loading. Please reload this page.