Skip to content

feat(config): Add postChecks toggle to disable GitHub Check-run posting - #458

Open
CalebKAston wants to merge 4 commits into
getsentry:mainfrom
babylist:caston/warden-post-checks-toggle-upstream
Open

feat(config): Add postChecks toggle to disable GitHub Check-run posting#458
CalebKAston wants to merge 4 commits into
getsentry:mainfrom
babylist:caston/warden-post-checks-toggle-upstream

Conversation

@CalebKAston

Copy link
Copy Markdown
Contributor

Summary

Adds a postChecks boolean (action input post-checks, default true; overridable per-repo via warden.toml's [defaults] postChecks) that disables creation/update of GitHub Check runs — the core warden check and every per-skill warden: <skill> check, including their inline annotations — while leaving PR review comment posting untouched.

Why

Check-run creation is entirely write-only today: nothing in the codebase reads it back via the API, and it isn't load-bearing for the findings-output file or for comment dedup (which reads back PR review comments instead, a separate mechanism). Some teams want PR review comments as the single review surface without a second, non-threaded, non-dedup'd GitHub Checks UI surface running alongside it — this makes that toggleable without losing the option to fail a required status check via fail-check/fail-on when wanted.

Implementation

  • Single choke point: checkOptionsForPullRequest(context, postChecks) returns undefined when postChecks is false. Two call sites that previously bypassed it with inline option objects (setupGitHubState, finalizeWorkflow's core-check update) were refactored to route through it too, so every check-creation path in pr-workflow.ts shares one gate.
  • postChecks resolves with the same enforced-baseline precedence already used for runnerConcurrency/auxiliary-model settings in initializeWorkflow: org base config (via base-config-path) wins over the repo's own warden.toml, which wins over the post-checks action input default. This matters because it's the same class of workflow-level (not per-trigger) setting, and a repo-level override should not be able to silently defeat an org-wide policy set via the base config.
  • ActionInputs.postChecks is non-optional (unlike the default-false booleans requestChanges/failCheck) because its default is true and gets resolved at parse time rather than deferred to each call site — the opposite polarity needs the default applied once, explicitly.

Verification

Applied this change to a local fork (babylist/warden) and verified it against real PRs in a downstream consumer repo before opening this PR:

  • With post-checks unset (default true): unchanged existing behavior — checks and comments both post, as today.
  • With post-checks: false: confirmed via the GitHub API that no warden / warden: <skill> check runs were created on the PR, while a PR review comment still posted for a real finding.
  • With post-checks: true set explicitly at the repo level (overriding an org base-config default of false): confirmed the full check-run set (core + every configured skill) was created again, alongside the review comment.

Also added regression tests exercising postChecks: false end-to-end through runPRWorkflow (both legacy run mode and report mode) asserting checks.create/checks.update are never called while pulls.createReview still is, plus config/schema tests for the warden.toml override and its precedence against the org base config.

pnpm lint && pnpm build && pnpm test all pass (two unrelated, pre-existing flaky tests in cli/files.test.ts/cli/output/jsonl.test.ts fail only under full-suite parallel load and pass in isolation — not touched by this change).

CalebKAston and others added 2 commits July 28, 2026 15:42
Check-run creation is entirely write-only (nothing in Warden reads it
back, and it doesn't gate dedup or the findings-file build), so it's
safe to let it be switched off independently of PR review comments.
Defaults to true to preserve existing behavior; a defaults.postChecks
in warden.toml can override the post-checks action input.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
initializeWorkflow already treats runnerConcurrency and auxiliaryOptions
as enforced by the org base config, with repo config only filling in
what the base omits (documented inline right above where postChecks
was resolved). postChecks is the same class of workflow-level setting,
but was reading the already-merged (repo-wins) config instead, so a
repo's own warden.toml could silently defeat an org policy meant to
turn checks off fleet-wide. Proved by reverting the fix and watching
the new org-base-config test fail exactly as expected.

Also drops the dead `options.postChecks ?? true` fallback in
finalizeReportWorkflow — its only caller always supplies postChecks,
so the fallback could never actually run.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@CalebKAston
CalebKAston marked this pull request as ready for review July 28, 2026 22:12
@CalebKAston
CalebKAston marked this pull request as draft July 28, 2026 22:13
fight-me review of this PR caught that post-checks had no docs entry
in either the workflow guide or llms.txt, and more importantly that
the guide's own Required Status Checks section recommends requiring
the warden check — setting post-checks: false on such a repo would
silently stop that check from ever running again, permanently
blocking merges through the normal GitHub UI. Documents the input in
both places and adds an explicit warning against that combination.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@CalebKAston
CalebKAston marked this pull request as ready for review July 28, 2026 22:25
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant