Skip to content

ci: wire up the arcup shell test suite - #250

Closed
ygd58 wants to merge 1 commit into
circlefin:mainfrom
ygd58:ci/run-arcup-shell-tests-248
Closed

ci: wire up the arcup shell test suite#250
ygd58 wants to merge 1 commit into
circlefin:mainfrom
ygd58:ci/run-arcup-shell-tests-248

Conversation

@ygd58

@ygd58 ygd58 commented Aug 10, 2026

Copy link
Copy Markdown

Summary

arcup/test_arcup.sh (829 lines, 24 assertions) currently passes but is never invoked anywhere -- not by CI, not by the Makefile, not by any other script. It already covers exactly the surface area where arcup's externally reported defects have concentrated (#204, #205): version comparison, checksum verification, archive extraction safety (symlink/path-traversal rejection), and download fallback behavior.

What this does not claim

The suite passes on main today and does not currently have assertions that catch #204 or #205 -- those need new test cases in a follow-up, per the issue. This PR only makes the existing suite reachable.

Changes

  • Makefile: new test-arcup target (also added to test-all)
  • .github/workflows/ci.yml: new arcup-test job -- no Rust toolchain or network needed, just checkout + bash arcup/test_arcup.sh, styled after the lightweight proto job

Testing

$ make test-arcup
... 24/24 ok

Note: test-finalize-release.sh, the other suite named in #248, is already covered by #249 -- this PR only handles the arcup half.

Fixes #248

arcup/test_arcup.sh (829 lines, 24 assertions) has never been run by
CI, the Makefile, or any other script. It exercises exactly the areas
where arcup's externally reported defects have concentrated (circlefin#204,
circlefin#205): version comparison, checksum/signature verification, archive
extraction (symlink/path-traversal rejection), and download fallback
behavior.

This does not itself fix circlefin#204 or circlefin#205 -- the suite passes on main
today and does not yet have assertions covering those two specific
defects. It just makes the suite reachable, so:
- `make test-arcup` runs it locally (also wired into `make test-all`)
- a new `arcup-test` CI job runs it in CI, on every PR

The job needs no Rust toolchain and no network, so it's just a
checkout + bash invocation; matches the lightweight style of the
existing `proto` job as the nearest example.

test-finalize-release.sh, the other suite named in circlefin#248, is already
covered by circlefin#249.

Fixes circlefin#248

@osr21 osr21 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked this against the current state of the tracker and against a fresh clone, and the difficult-but-honest verdict is that this ground is already claimed — twice over, with the split explicitly settled in the issue this PR says it fixes.

Timeline matters here:

  1. #247 (wolfgang1211, Aug 9) wires this exact suite into CI: make test-arcup + test-all + a dedicated job — and additionally runs shellcheck over arcup/arcup and arcup/install.
  2. #249 originally also covered the arcup half, and after the overlap was flagged on #248, its author rescoped it away specifically to concede the arcup ground to #247 — that's the settlement recorded in the #248/#249 threads. This PR re-enters the ground that rescope just vacated, and the note in your description ("#249 covers the finalize-release half") reflects the post-rescope state, which is exactly the state in which #247 covers this half.

Mechanical conflicts with #247, verified by diffing both against main:

  • Same ci.yml anchor (immediately after the docker job, line-158 context) — whichever merges second conflicts.
  • Both PRs edit the same test-all line in the Makefile.
  • If both somehow merged, the suite would run twice per CI pass.

Where #247 is a superset: its test-arcup target uses glob discovery (arcup/test_*.sh / arcup/*_test.sh with an unmatched-glob guard), so future suites are picked up with no CI change, and it adds shellcheck lint that this PR lacks. This PR's target hardcodes the single filename.

One factual correction: the suite has 25 assertions on main, not 24 — I ran it on both main and this branch just now (identical output, 25 ok lines, exit 0). Worth fixing if any part of this survives, since reviewers use that number as a checksum of "did it actually run."

Constructive redirect, because the energy is pointed at a real gap: the genuinely unclaimed follow-up named in #248 is new assertions that would have caught #204 (self-update authenticity) and #205 (version_gt prerelease-vs-stable). Nobody has opened that PR. It's pure test-writing against arcup/test_arcup.sh, it composes with #247 rather than colliding (that file is exactly what #247 deliberately doesn't touch), and it converts this suite from "existing coverage now enforced" into coverage that pins two live bugs' fixes when they land. That contribution would be additive instead of third-in-line.

@ygd58

ygd58 commented Aug 10, 2026

Copy link
Copy Markdown
Author

Fair verdict, thanks for the careful diff. Confirmed 25 assertions on a fresh main pull (my 24 was stale/miscounted) and confirmed #247's superset: glob discovery + shellcheck vs. this PR's hardcoded filename. Closing in favor of #247.

On the constructive redirect — checked the two PRs that fix #204/#205 directly, and that ground turns out to already be claimed too, just not yet merged: #212 (danilaverbena) adds 25 assertions covering the full SemVer 11.4 precedence chain plus build-metadata rules, and #223 (zkasuran) adds authenticity tests (tampered installer, checksum mismatch, fail-closed without a published release, etc.) alongside the fix itself. Both already answer 'does this suite catch #204/#205' once they land, so a standalone test-only PR would duplicate their work rather than fill a gap.

@ygd58 ygd58 closed this Aug 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: the arcup and finalize-release shell test suites are never run

2 participants