Skip to content

test(ci): pin reusable-workflow policy and guardrail negative controls - #13

Open
SaulBuilds wants to merge 1 commit into
mainfrom
dgx/qg-pins-1022
Open

SaulBuilds wants to merge 1 commit into
mainfrom
dgx/qg-pins-1022

Conversation

@SaulBuilds

Copy link
Copy Markdown
Contributor

Tests only; no workflow behaviour changes. These are org-wide templates, so this PR adds checks rather than edits.

What

  • scripts/ci/test_workflow_policy.py (stdlib unittest, no network), run from ci.yml:
    • Negative controls for workflow-guardrails.sh: each check (C1 misplaced workflow, C2 continue-on-error / || true, C3 mutable branch pin / mutable cross-repo ref, C4 ${{ }} in run:) is run against a mutated temp copy of the tree and must report FAIL. The clean tree must PASS.
    • Policy pins the guardrail doesn't cover: blocking dependency audit in the rust/js/python pipelines (on by default); callable, checksum-pinned secret scan; a missing requested JS script fails the job; package-manager allowlist; Slither on by default; every remote action SHA-pinned; Dependabot config in .github/; caller snippets pinned to a SHA; install lines constrained to --proto '=https' --tlsv1.2; a reachable security-report channel; a README row for each reusable workflow.

Standard: met

  • Revert check: each guarded property was reverted one at a time in a scratch tree, 14 in all. Every one turns a named test red. One of them (the README table) was green until I tightened the test to match table rows instead of any mention.
  • Suite: 19 tests OK. workflow-guardrails.sh PASS. test-canon-guardrail.sh PASS.
  • No network and no host side effects: temp dirs only.

Left open (owner)

  • reusable-canon-guardrail.yml checks out the shared script at a fixed SHA that predates the path-only exclusion fix, so callers still run the older script. The fix is to bump that ref: to a post-fix SHA. That changes what runs in every caller, so I'm leaving it for a deliberate bump.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KGMeatV8UjtCB4CTbWuAv4

Adds scripts/ci/test_workflow_policy.py (stdlib unittest, no network) and runs
it from ci.yml:

- negative controls for workflow-guardrails.sh: each check (C1 misplaced
  workflow, C2 continue-on-error / exit-code swallow, C3 mutable branch pin and
  mutable cross-repo ref, C4 template expansion in run:) is fed a mutated copy
  of the workflow tree and must report FAIL; the clean tree must PASS.
- policy pins the guardrail does not look at: blocking dependency audit in the
  rust/js/python pipelines (default on), callable checksum-pinned secret scan,
  missing requested js script fails the job, package-manager allowlist,
  Slither on by default, every remote action SHA-pinned, Dependabot config in
  .github/, caller snippets pinned to a SHA, protocol-constrained install
  lines, reachable security-report channel, README row per reusable workflow.

Revert check: each guarded property was reverted one at a time in a scratch
tree; every one turns a named test red. No workflow behaviour changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KGMeatV8UjtCB4CTbWuAv4

@BerryManifold BerryManifold left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Codex static review of 8e8f8b0a92f120ef6482ee1acf3141466d0c1760 is complete. Result: clean; details retained privately. Details and coverage limits are recorded in the project's private review log. No tests were executed by this reviewer. Both account submissions represent the same Codex review, not independent reviews.

@Matr0xshka Matr0xshka left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Codex static review of 8e8f8b0a92f120ef6482ee1acf3141466d0c1760 is complete. Result: clean; details retained privately. Details and coverage limits are recorded in the project's private review log. No tests were executed by this reviewer. Both account submissions represent the same Codex review, not independent reviews.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants