[#2870, #2871, #2872] Pinned the seed script, 'asciinema' and 'sharp-cli' versions. - #2874
Conversation
WalkthroughThe test asset updater now pins the ChangesReproducibility hardening
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
2 similar comments
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2874 +/- ##
==========================================
- Coverage 86.81% 86.38% -0.43%
==========================================
Files 100 93 -7
Lines 4846 4687 -159
Branches 47 3 -44
==========================================
- Hits 4207 4049 -158
+ Misses 639 638 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
📖 Documentation preview for this pull request has been deployed to Netlify: https://6a6c18ed465477ff07661fe2--vortex-docs.netlify.app This preview is rebuilt on every commit and is not the production documentation site. |
…r of manual bumps.
…ntation video workflows.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
.vortex/tests/update-test-assets (1)
239-240: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winPin the Alpine image tag.
alpineresolves a mutable image version. This can change the cleanup behavior between runs and violates the reproducibility objective. Usealpine:3.24or an immutable digest.Proposed fix
- 'docker run --rm -v %s:/sut alpine find /sut -mindepth 1 -delete', + 'docker run --rm -v %s:/sut alpine:3.24 find /sut -mindepth 1 -delete',🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.vortex/tests/update-test-assets around lines 239 - 240, Update the Docker cleanup command in the test asset update script to use a pinned Alpine image, replacing the mutable alpine reference with alpine:3.24 or an immutable digest while preserving the existing volume mount and cleanup arguments.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In @.vortex/tests/update-test-assets:
- Around line 239-240: Update the Docker cleanup command in the test asset
update script to use a pinned Alpine image, replacing the mutable alpine
reference with alpine:3.24 or an immutable digest while preserving the existing
volume mount and cleanup arguments.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: e8155ed0-3a37-4761-a213-610995a007b7
📒 Files selected for processing (2)
.vortex/docs/content/contributing/maintenance/template.mdx.vortex/tests/update-test-assets
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
1 similar comment
This comment has been minimized.
This comment has been minimized.
|
Code coverage (threshold: 90%) Per-class coverage |
This comment has been minimized.
This comment has been minimized.
2 similar comments
This comment has been minimized.
This comment has been minimized.
|
Code coverage (threshold: 90%) Per-class coverage |
Closes #2872
Closes #2870
Closes #2871
Summary
Three tools were resolved fresh at run time rather than being pinned, so what actually executed depended on when a command ran instead of on what the commit said. Two of them are artifacts downloaded over the network and then executed, which makes this a supply-chain concern as well as a reproducibility one. All three produce committed output (database images and documentation videos), so an unnoticed upstream change lands as a diff in generated artifacts rather than as an obvious failure.
Each is now pinned by the mechanism that suits it: the two downloaded artifacts get an explicit release tag plus a recorded SHA-256 checksum, and the one that is an npm package is installed from the existing lockfile instead of ad hoc. A pinned tag alone is not enough for the downloads, because a GitHub release asset can be replaced without the tag moving.
Changes
Test asset seeding (
.vortex/tests/update-test-assets)SEED_URLmoves fromreleases/latest/downloadto the explicit26.7.0tag. That is the current release, and the one that fixed arm64 seeding, so behaviour is unchanged today.SEED_SHA256records the hash of that release'sseed.sh, and a newverify_checksum()fails closed before the script is made executable and run.Documentation video workflows (
.github/workflows/vortex-release.yml,.github/workflows/vortex-test-installer.yml)asciinemamoves fromreleases/latest/downloadtov3.2.1, with its SHA-256 checked viasha256sum --check --strictbefore the binary is made executable. The version and checksum are declared in a step-levelenvblock and referenced as shell variables rather than interpolated into therunscript, which keeps the step free of template-injection warnings.sharp-cliis no longer installed globally withnpm i -g. It is now a declared dependency of.vortex/docs, so it is pinned byyarn.lockwith integrity hashes and installed by theyarn install --frozen-lockfilethe step already runs. The global install is deleted from both workflows.asciinemavalues so the release build and the installer test always generate assets with the same tooling.Video recorder (
.vortex/docs/.utils/VideoRecorder.php)npx sharp-clitonpx sharp. Thesharp-clipackage installs its executable under the namesharp, so with a local install there is nosharp-clibinary innode_modules/.binandnpx sharp-cliwould fall back to fetching the package from the registry, which is the behaviour this change is removing.Documentation
Notes for reviewers
sharp-cliwas originally pinned here asnpm i -g sharp-cli@5.2.0, which zizmor flagged asadhoc-packages("installs a package outside of a lockfile") on both workflows. Moving it into the docspackage.jsonresolves that finding at the source rather than suppressing it, and gives a stronger guarantee than a version pin sinceyarn.lockalso carries integrity hashes. It matches howsvg-term-cli, the sibling tool used by the same recorder, is already declared. The lockfile carries the@img/sharp-linux*binaries, so the Linux runners resolve native builds from it.Two of the issues suggested using Renovate to keep these pins from rotting. That was not done: all of these are bumped by hand. Worth knowing if that is ever revisited, because the cases are not equivalent:
.github/workflows/, which Renovate can already see, so acustomManagersentry there would work as the issue describes..vortex/tests/update-test-assetscannot be tracked.renovate.jsonsetsignorePathsto.vortex/**, and Renovate appliesignorePathsto the global file list before any manager runs, custom regex managers included, with no negation syntax available. A custom manager pointed at that path would be inert, and making it work would mean widening Renovate's surface across.vortex/.ignorePathsentry covers.vortex/docs, so the newsharp-clilockfile entry is not Renovate-tracked either. It is still an improvement: the version is now resolved from a committed lockfile rather than from whatever npm publishes at run time.Screenshots
N/A
Before / After