ci: wire up the arcup shell test suite - #250
Conversation
arcup/test_arcup.sh (829 lines, 24 assertions) has never been run by CI, the Makefile, or any other script. It exercises exactly the areas where arcup's externally reported defects have concentrated (circlefin#204, circlefin#205): version comparison, checksum/signature verification, archive extraction (symlink/path-traversal rejection), and download fallback behavior. This does not itself fix circlefin#204 or circlefin#205 -- the suite passes on main today and does not yet have assertions covering those two specific defects. It just makes the suite reachable, so: - `make test-arcup` runs it locally (also wired into `make test-all`) - a new `arcup-test` CI job runs it in CI, on every PR The job needs no Rust toolchain and no network, so it's just a checkout + bash invocation; matches the lightweight style of the existing `proto` job as the nearest example. test-finalize-release.sh, the other suite named in circlefin#248, is already covered by circlefin#249. Fixes circlefin#248
osr21
left a comment
There was a problem hiding this comment.
Checked this against the current state of the tracker and against a fresh clone, and the difficult-but-honest verdict is that this ground is already claimed — twice over, with the split explicitly settled in the issue this PR says it fixes.
Timeline matters here:
- #247 (wolfgang1211, Aug 9) wires this exact suite into CI:
make test-arcup+test-all+ a dedicated job — and additionally runsshellcheckoverarcup/arcupandarcup/install. - #249 originally also covered the arcup half, and after the overlap was flagged on #248, its author rescoped it away specifically to concede the arcup ground to #247 — that's the settlement recorded in the #248/#249 threads. This PR re-enters the ground that rescope just vacated, and the note in your description ("#249 covers the finalize-release half") reflects the post-rescope state, which is exactly the state in which #247 covers this half.
Mechanical conflicts with #247, verified by diffing both against main:
- Same
ci.ymlanchor (immediately after the docker job, line-158 context) — whichever merges second conflicts. - Both PRs edit the same
test-allline in the Makefile. - If both somehow merged, the suite would run twice per CI pass.
Where #247 is a superset: its test-arcup target uses glob discovery (arcup/test_*.sh / arcup/*_test.sh with an unmatched-glob guard), so future suites are picked up with no CI change, and it adds shellcheck lint that this PR lacks. This PR's target hardcodes the single filename.
One factual correction: the suite has 25 assertions on main, not 24 — I ran it on both main and this branch just now (identical output, 25 ok lines, exit 0). Worth fixing if any part of this survives, since reviewers use that number as a checksum of "did it actually run."
Constructive redirect, because the energy is pointed at a real gap: the genuinely unclaimed follow-up named in #248 is new assertions that would have caught #204 (self-update authenticity) and #205 (version_gt prerelease-vs-stable). Nobody has opened that PR. It's pure test-writing against arcup/test_arcup.sh, it composes with #247 rather than colliding (that file is exactly what #247 deliberately doesn't touch), and it converts this suite from "existing coverage now enforced" into coverage that pins two live bugs' fixes when they land. That contribution would be additive instead of third-in-line.
|
Fair verdict, thanks for the careful diff. Confirmed 25 assertions on a fresh main pull (my 24 was stale/miscounted) and confirmed #247's superset: glob discovery + shellcheck vs. this PR's hardcoded filename. Closing in favor of #247. On the constructive redirect — checked the two PRs that fix #204/#205 directly, and that ground turns out to already be claimed too, just not yet merged: #212 (danilaverbena) adds 25 assertions covering the full SemVer 11.4 precedence chain plus build-metadata rules, and #223 (zkasuran) adds authenticity tests (tampered installer, checksum mismatch, fail-closed without a published release, etc.) alongside the fix itself. Both already answer 'does this suite catch #204/#205' once they land, so a standalone test-only PR would duplicate their work rather than fill a gap. |
Summary
arcup/test_arcup.sh(829 lines, 24 assertions) currently passes but is never invoked anywhere -- not by CI, not by the Makefile, not by any other script. It already covers exactly the surface area where arcup's externally reported defects have concentrated (#204, #205): version comparison, checksum verification, archive extraction safety (symlink/path-traversal rejection), and download fallback behavior.What this does not claim
The suite passes on
maintoday and does not currently have assertions that catch #204 or #205 -- those need new test cases in a follow-up, per the issue. This PR only makes the existing suite reachable.Changes
Makefile: newtest-arcuptarget (also added totest-all).github/workflows/ci.yml: newarcup-testjob -- no Rust toolchain or network needed, just checkout +bash arcup/test_arcup.sh, styled after the lightweightprotojobTesting
Note:
test-finalize-release.sh, the other suite named in #248, is already covered by #249 -- this PR only handles thearcuphalf.Fixes #248