Skip to content

test: run the real code instead of hand-copied mirrors (sable-1drb) - #225

Closed
Rome-1 wants to merge 1 commit into
fix/sable-fgk7-unreadable-report-is-not-cleanfrom
fix/sable-1drb-tests-run-the-real-code
Closed

test: run the real code instead of hand-copied mirrors (sable-1drb)#225
Rome-1 wants to merge 1 commit into
fix/sable-fgk7-unreadable-report-is-not-cleanfrom
fix/sable-1drb-tests-run-the-real-code

Conversation

@Rome-1

@Rome-1 Rome-1 commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Summary

Stacked on #224 (base branch is fix/sable-fgk7-unreadable-report-is-not-clean; retarget to main once that merges).

Two test suites passed by testing a transcription of the code rather than the code. Both now run the real thing.

GitHub Action bash (sable-1drb). test-threshold-eval.sh and test-pr-comment-tip.sh each carried their own copy of the severity case statement and the report-only tip block, hand-copied from action.yml with a header admitting it must be kept in sync by hand. 39 assertions that could pass in full while action.yml was broken, and would keep passing after anyone edited it. The logic now lives once in github-action/lib/severity.sh (rafter_threshold_fails, rafter_report_only_tip), sourced by both action.yml steps via ${{ github.action_path }} and by both tests. Behaviour is unchanged; the same 31 + 10 cases pass, plus two new ones pinning the ::warning:: on an unknown threshold and the tip naming severity-threshold: high.

Node issues.test.ts (specimen 11 of sable-d2x2, found while fixing #224). The file mirrored four modules (dedup, issue-builder, from-text, from-scan) and tested the mirrors; one had already drifted (the copied draftsFromLocalScan parsed a bare array only, while the real one has accepted {results:[...]} since 0.7.8). The 172 lines of copies are gone; every describe now imports the shipped function. Three source functions gained export for that (severityLabel, parseNaturalText, draftsFromLocalScan); no behaviour change.

Coverage that proves the wiring, not just the logic

  • New end-to-end job test-threshold-gate-end-to-end: the real action against the mock with 1 critical, 1 high, 1 low and severity-threshold: high. Asserts status=completed and outcome=failure and findings-count=3 — the gate's signature: the report was read and counted, then the threshold rejected it. A unit test on the library cannot prove action.yml calls it; this does.
  • Drift detector: checks 3 and 4 now read lib/severity.sh; new check 17 asserts action.yml sources the library in both steps, calls both functions, and carries no inline case branch, so a "simplification" that pastes the case statement back into the YAML cannot land green and silently detach the tests again.

Mutation-verified by hand (sable-d2x2 rule A, failing test names in the log):

  • Library: high) made to ignore high → 2 threshold cases fail. none) set to fail → drift check 4 fails and 6 threshold cases fail. Tip gate dropped → 3 tip cases and drift check 3 fail.
  • action.yml: one source line removed → check 17 fails. An inline critical) branch added → check 17 fails.
  • Node: severityLabel mapping error→high → 5 tests fail. draftsFromLocalScan returning [] → 6 tests fail. Before this PR, both mutations passed the suite untouched, because the suite never called them.

Local: drift detector 0 failures; both YAML files parse; tsc --noEmit clean; issues.test.ts 109 pass; bash -e sanity that rafter_report_only_tip with no findings does not trip errexit inside the comment builder.

The action's threshold-eval and PR-comment-tip tests each carried their
own transcription of the bash they tested, so they could pass in full while
action.yml was broken. The logic now lives once in
github-action/lib/severity.sh, sourced by both action.yml steps and by both
tests. Behaviour unchanged. A new end-to-end job runs the real action with
findings above the threshold and asserts status=completed AND
outcome=failure, which proves the counts reach the gate; drift check 17
asserts action.yml sources the library in both steps and carries no inline
copy, so the tests cannot be silently detached again.

node/tests/issues.test.ts mirrored four modules (dedup, issue-builder,
from-text, from-scan) and tested the mirrors; one had already drifted. The
copies are removed and every describe imports the shipped function. Three
source functions gained `export` for that.

Every guard was mutated by hand: each mutation fails named tests that were
untouched by the same mutation before this change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant