ci: the arcup and finalize-release shell test suites are never run

[mehmetkr-31]

Summary

The repo ships two shell test suites and neither is executed anywhere — not by CI, not by the Makefile, not by any other script.

Suite	Size	Status today
arcup/test_arcup.sh	829 lines, 25 assertions	passes, exit 0
.github/scripts/test-finalize-release.sh	65 lines	passes, exit 0

$ grep -rn "test_arcup" --include="*" . | grep -v "^./arcup/test_arcup.sh"
(no matches)
$ grep -rn "test-finalize-release" .github/ Makefile | grep -v "^.github/scripts/test-finalize-release.sh"
(no matches)

Both run in seconds and need no network — test_arcup.sh sources arcup with ARCUP_SKIP_MAIN=1 and drives it against local fixtures.

Why it matters for arcup specifically

arcup is the installer users actually run, and it is where the externally reported defects have concentrated: #45 (merged), plus #204 and #205 still open. Its suite already covers those exact areas:

• compares prerelease installer versions and same prerelease base is not newer — the ground Bug: arcup SemVer comparison treats prerelease and stable versions as equal #205 (version_gt treating prerelease and stable as equal) sits on.
• install_binary rejects symlink, archive path traversal fails, archive link entries fail, rejects non-https GitHub API URL, valid checksum file passes — the ground Security: arcup --self-update replaces the installer without authenticity verification #204 (self-update without authenticity verification) sits on.

Worth being precise about what this does and does not claim: the suite passes on main today, so it does not currently catch #204 or #205 — those need new assertions. The point is that a regression test added alongside either fix would never run, which makes writing one pointless until a job exists.

Suggested fix

A shell-tests job running both suites. No toolchain needed, so it is cheap and can run in parallel with everything else. Opened as #249.

The suites are written to be portable — arcup selects between sha256sum and shasum, and test_arcup.sh branches on GNU vs BSD tar when building its path-traversal fixture — but I verified them on macOS only, so the job's first run on ubuntu-latest is the confirming check. The failure mode is a red job rather than anything silent.

[osr21]

Verified the premise independently on a fresh clone of main, and one coordination point worth flagging.

The diagnosis is correct. grep -rn "test_arcup" --exclude-dir=.git and the same for test-finalize-release both come back empty outside the scripts themselves — neither suite is referenced by CI, the Makefile, or anything else.

Your macOS-only caveat can be closed. I ran both suites on Linux (GNU coreutils/tar, bash 5): arcup/test_arcup.sh passes all 25 assertions and .github/scripts/test-finalize-release.sh prints finalize-release tests passed, both exit 0. The portability branching (sha256sum/shasum, GNU vs BSD tar) works as intended, so the first ubuntu-latest run should be green.

The coordination point: #247 (opened a couple of hours before this issue) already wires the arcup half. wolfgang1211's PR adds an Arcup Installer CI job plus a make test-arcup target (glob-based, picks up future arcup/*_test.sh files) and runs shellcheck over arcup/arcup and arcup/install. I compared the two diffs: #247 and #249 insert their jobs at the exact same anchor in ci.yml (after the docker job, line 158), so whichever merges second gets a guaranteed conflict, and if both merged the arcup suite would run twice per CI pass.

The two PRs aren't redundant though — each has a piece the other lacks:

	#247	#249
arcup suite in CI	✅	✅ (duplicate)
make test-arcup + test-all wiring	✅	—
shellcheck lint	✅	—
test-finalize-release.sh in CI	—	✅ (unique)

The clean resolution seems to be: let #247 carry the arcup half (it's strictly a superset there — Makefile target, test-all integration, lint), and slim #249 down to the genuinely novel part, the finalize-release suite — either as its own small job or as one extra step contributed to #247's job. That keeps both contributions intact, avoids the conflict, and gets everything runnable.

One reason to care about landing this soon regardless of how it's split: #243 adds new assertions to arcup/test_arcup.sh for a checksum-parsing fix, and until one of these CI PRs merges, those regression tests are exactly what you called them — decorative.

[ygd58]

Opened #250 for the arcup half — a new arcup-test CI job plus a make test-arcup target, styled after the existing lightweight proto job (no Rust toolchain or network needed). Scoped it to not claim it catches #204/#205 yet, since the suite passes as-is today; that would need new assertions in a follow-up. The test-finalize-release.sh half is already covered by #249.

[osr21]

@ygd58 The arcup half is the piece that was already claimed — that was the point of the coordination comment above. #247 (opened Aug 9, before this issue existed) wires the same suite with a superset of #250's changes: glob-based make test-arcup (future arcup/*_test.sh files picked up with no CI change), test-all integration, and shellcheck over arcup/arcup + arcup/install. #249's rescope conceded that ground to #247 specifically so only one PR would carry it; #250 re-enters it as a third claimant and conflicts with #247 on both files (same ci.yml anchor, same test-all line). Full details, including one factual correction (the suite has 25 assertions, not 24 — verified on main and on your branch), are in my review on #250.

The half of this issue that genuinely has nobody on it is the one you scoped out: the new assertions that would pin #204 (self-update authenticity) and #205 (version_gt prerelease-vs-stable). That's pure test-writing against arcup/test_arcup.sh — the exact file #247 deliberately doesn't touch — so it composes with everything in flight instead of colliding, and it's the piece that turns this wiring from "existing coverage enforced" into coverage that actually guards two live bugs' fixes. If you want work here that's additive rather than third-in-line, that's the PR to open.

Current state of the split, for anyone landing on this issue fresh: arcup CI+lint → #247; finalize-release CI → #249 (rescoped); #204/#205 regression assertions → unclaimed; #250 → duplicates #247.

[ygd58]

Cross-linking so this does not look unclaimed to the next person landing here: I checked the same ground right after your #250 review and left this on that thread, but repeating here since it belongs on this issue too.

#212 (danilaverbena, fixing #205) already adds 25 assertions in a new arcup/version_gt_test.sh covering the full SemVer 11.4 precedence chain (alpha < alpha.1 < beta.N < rc < stable) plus build-metadata rules -- including the exact stable-vs-own-prerelease case (version_gt("0.3.0", "0.3.0-rc.1")) that the current test_arcup.sh on main does not exercise (it only checks the reverse direction).

#223 (zkasuran, fixing #204) adds authenticity-verification assertions alongside its fix -- tampered installer rejected, checksum mismatch reported, fails closed without a published release, etc.

So the #204/#205 regression-assertion ground already has two claimants, each carrying tests alongside their own fix rather than as a standalone test-only PR. Closed #250. If either #212 or #223 ends up superseded before merge, the assertions would need re-homing, but as of today neither is actually open ground.

[osr21]

Confirmed all of it independently — and closing #250 the moment the picture changed was the right move, cleanly done.

Verified on #212's branch: arcup/version_gt_test.sh runs green (version_gt: 25 passed, 0 failed) and contains exactly the case you named — assert_gt "0.3.0" "0.3.0-rc.1" at line 42, with the reverse alongside. And you're right about main's gap: test_arcup.sh today only tests the reverse direction (0.3.0-rc.1 not newer than 0.3.0), which passes even under the broken equal-treatment version_gt — the stable-beats-own-prerelease direction is the discriminating assertion, and only #212 has it.

Verified on #223: it carries +235/-2 on arcup/test_arcup.sh alongside the fix — authenticity assertions living with the code they pin, as you said.

Two composition facts worth adding to the record, since they connect these PRs to the CI wiring in ways nobody has stated yet:

1. fix(arcup): honor SemVer prerelease precedence in version_gt (fixes #205) #212's test filename already matches ci: run arcup installer shell tests and shellcheck #247's discovery glob. version_gt_test.sh fits arcup/*_test.sh, so the moment ci: run arcup installer shell tests and shellcheck #247 lands, fix(arcup): honor SemVer prerelease precedence in version_gt (fixes #205) #212's suite gets picked up by make test-arcup and CI with zero additional wiring — no edit to either PR. That's the glob-discovery design paying for itself on the first real case, and incidentally the concrete difference between ci: run arcup installer shell tests and shellcheck #247 and the hardcoded-filename approach.

2. One genuine conflict remains in the updated map: fix(arcup): verify the installer before self-update replaces it #223 vs fix(arcup): accept checksum files without a trailing newline #243. Both modify arcup/test_arcup.sh, and fix(arcup): verify the installer before self-update replaces it #223's +235/-2 rework touches the checksum-verification area where fix(arcup): accept checksum files without a trailing newline #243 (mine) adds its no-trailing-newline assertions. They fix different bugs at different layers (fix(arcup): verify the installer before self-update replaces it #223: release authenticity; fix(arcup): accept checksum files without a trailing newline #243: checksum-file parsing), so both belong — but whichever merges second likely needs a mechanical rebase in that file. Flagging it now so it's an expected rebase, not a surprise.

Updated state of the split: arcup CI+lint → #247; finalize-release CI → #249; #205 assertions → #212 (with fix); #204 assertions → #223 (with fix); #243 → checksum parsing (rebase coordination with #223 noted); #250 → closed. Every piece of #248's ground is now claimed exactly once — this issue is fully dispatched pending merges.