diff --git a/.github/DESKTOP_NIGHTLY.md b/.github/DESKTOP_NIGHTLY.md index 854ede602d..8cfaf33e83 100644 --- a/.github/DESKTOP_NIGHTLY.md +++ b/.github/DESKTOP_NIGHTLY.md @@ -21,7 +21,7 @@ Desktop Nightly is an ephemeral developer snapshot, not an Apache release. It builds the current `main` commit every day so contributors can try recent Desktop changes and report problems without waiting for an ASF source-release vote. -The npm publication workflow gives each snapshot an immutable version such as `0.2.0-dev.42.20260829`. The run number is the sole ordering authority. After that exact npm version is public, it triggers Desktop Nightly with a version-only artifact; the authenticated workflow event supplies the exact source commit and upstream run. Each fresh Desktop Nightly creates a `v` tag protected by the checked-in `Immutable release tags` ruleset and one GitHub draft prerelease containing the macOS and Windows packages, blockmaps, `dev-mac.yml`, `dev.yml`, and one offline Sigstore bundle. The workflow verifies every remote asset before it publishes the prerelease as non-Latest. Packaged Nightlies use the GitHub `dev` channel and verify that downloaded bytes were attested by `.github/workflows/desktop-nightly.yml` on `main`. A formal Desktop build uses the separate stable GitHub Release channel and formal product-release attestation identity. +The npm publication workflow gives each snapshot an immutable version such as `0.2.0-dev.42.20260829`. The run number is the sole ordering authority. After that exact npm version is public, it triggers Desktop Nightly with a version-only artifact; the authenticated workflow event supplies the exact source commit and upstream run. Each fresh Desktop Nightly creates a `v` tag protected by the checked-in `Immutable release tags` ruleset and one GitHub draft prerelease containing the macOS, Windows, and Linux packages, blockmaps, `dev-mac.yml`, `dev.yml`, `dev-linux.yml`, `dev-linux-arm64.yml`, and one offline Sigstore bundle. macOS builds one architecture per runner and both write the same feed name, so each upload carries `dev-mac-.yml` and publication merges them into the single `dev-mac.yml` clients read. The workflow verifies every remote asset before it publishes the prerelease as non-Latest. Packaged Nightlies use the GitHub `dev` channel and verify that downloaded bytes were attested by `.github/workflows/desktop-nightly.yml` on `main`. A formal Desktop build uses the separate stable GitHub Release channel and formal product-release attestation identity. Nightly currently uses the same application identity as the formal Desktop. Installing it replaces the existing Maka installation rather than creating a second side-by-side app. Its user data remains in the same location. Testers who need the formal build should reinstall that build before returning to the formal channel. Builds previously downloaded from `nightlies.apache.org` do not migrate automatically; testers must install the newest GitHub prerelease once, after which GitHub Nightlies update automatically. @@ -31,8 +31,8 @@ Nightly currently uses the same application identity as the formal Desktop. Inst 2. Configure npm Trusted Publishing for `apache/maka` and `.github/workflows/npm-publication.yml`, restricted to the `npm-publication` Environment and with both `npm publish` and `npm stage publish` allowed. Do not create or store a long-lived npm token. 3. After npm Trusted Publishing is ready, set `NPM_NIGHTLY_ENABLED` to `true`, run `npm publication` from `main` with `channel=nightly`, and verify the exact npm version and `nightly` dist-tag. 4. Set `DESKTOP_NIGHTLY_ENABLED` to `true` and manually dispatch a fresh npm Nightly. Confirm that its successful run triggers `Desktop Nightly`. Do not rerun a failed attempt in place. -5. Verify that `v` points to the exact source SHA and that its GitHub Release is published with Draft off, Prerelease on, Latest off, and exactly the nine expected assets. Install that prerelease on both platforms. -6. Publish one later fresh Nightly and confirm a GitHub-to-GitHub automatic and differential update on both platforms before sharing the channel with testers. +5. Verify that `v` points to the exact source SHA and that its GitHub Release is published with Draft off, Prerelease on, Latest off, and exactly the expected assets, which `desktopNightlyReleaseAssetNames` defines. Install that prerelease on every packaged target. +6. Publish one later fresh Nightly and confirm a GitHub-to-GitHub automatic and differential update on every packaged target before sharing the channel with testers. Linux AppImage installs update in place; Linux deb installs ask for the privilege `dpkg` needs. The npm schedule starts at 18:17 UTC. Before changing the npm tag, the workflow requires its run number to exceed the current `nightly` version. Desktop assembles and verifies a draft before one publish mutation; a packaging, attestation, tag, upload, or digest failure leaves no partially published GitHub Release. Never rerun a failed workflow attempt in place; dispatch a fresh npm Nightly with a newer version. diff --git a/.github/RELEASE_CHECKLIST.md b/.github/RELEASE_CHECKLIST.md index 4402722337..de76baf582 100644 --- a/.github/RELEASE_CHECKLIST.md +++ b/.github/RELEASE_CHECKLIST.md @@ -27,8 +27,9 @@ gate. The workflow creates no Draft until every required artifact job succeeds. Phase 1 requires: -- signed and notarized Apple Silicon macOS Desktop artifacts; +- signed and notarized Apple Silicon and Intel macOS Desktop artifacts; - the unsigned Windows x64 Desktop installer and ZIP; +- the unsigned x64 and arm64 Linux AppImage and deb; - the signed, notarized, relocatable Apple Silicon CLI/TUI ZIP; - checksums generated after each artifact reaches its final form. @@ -84,12 +85,12 @@ bytes and expected filename are not covered by that protected workflow identity. version, and the CLI manifest exposes only the `maka` command. 4. Dispatch `Release` from the exact approved candidate tag and supply the same tag as `source_reference_tag`. A rerun must use that same tag; never select current `main` instead. -5. Confirm `release-identity`, both Desktop matrix entries, `cli-macos-arm64`, and +5. Confirm `release-identity`, every Desktop matrix entry, `cli-macos-arm64`, and `publish` pass. A skipped or failed required job must prevent Draft creation. 6. Confirm one Draft named `v` targets the approved source SHA, identifies the ASF source reference in its notes, is not marked as a GitHub prerelease or Latest while it remains a Draft, - and contains exactly the manifest - reported by `node scripts/product-release-artifacts.mjs list`. The manifest covers both Desktop + and contains exactly the release asset names + reported by `node scripts/product-release-artifacts.mjs list`. That list covers all three Desktop platforms and update metadata, the standalone CLI/TUI, and their required checksums. 7. Inspect the CLI ZIP. It must contain `bin/maka`, `RELEASE.json`, `DISCLAIMER-WIP`, `LICENSE`, `NOTICE`, `THIRD_PARTY_NOTICES.txt`, the pinned Node license, and no `bin/maka-agent`. @@ -100,7 +101,7 @@ bytes and expected filename are not covered by that protected workflow identity. entitlements must retain the required hardened-runtime capabilities and omit `com.apple.security.get-task-allow`, as required by Apple's [notarization guidance](https://developer.apple.com/documentation/security/resolving-common-notarization-issues). -9. Inspect both Desktop resource trees and confirm `git/`, `bundled-git.json`, `licenses/git/`, and +9. Inspect every Desktop resource tree and confirm `git/`, `bundled-git.json`, `licenses/git/`, and `licenses/dugite/` are absent. If the publish job created the product tag or Draft but failed before every asset was uploaded, @@ -131,7 +132,11 @@ stable release Latest in the same GitHub operation. Do not publish or change the Latest designation manually. A failed or rejected npm candidate requires a new product version; never publish the Draft to work around npm state. -## Acceptance on another Apple Silicon Mac +## Acceptance on another Mac + +Run this section twice: once on an Apple Silicon Mac with the `mac-arm64` DMG, and once on an Intel +Mac with the `mac-x64` DMG. The CLI/TUI ships for Apple Silicon only, so steps 4 to 7 belong to the +Apple Silicon pass. Download the DMG, CLI ZIP, and their checksum files through a browser from the Draft. Do not move artifacts directly from the workflow runner; the browser path supplies the real quarantine @@ -168,9 +173,26 @@ Download the installer, Windows Desktop ZIP, and both checksum files through a b 7. Add a clean remote Runtime Host from the packaged Desktop app. Confirm setup installs the exact public `maka-agent@` package and the remote session completes one model turn. +## Acceptance on a Linux machine + +Run this section twice: once on x64 with the `x86_64` AppImage and the `amd64` deb, and once on +arm64 with the `arm64` pair. Download both distributables and their checksum files through a browser +from the same Draft. + +1. Run `sha256sum -c` for the AppImage and the deb. +2. `chmod +x` the AppImage and launch it once to confirm the portable artifact starts. +3. Install the deb with `sudo apt install ./Maka--linux-.deb` and launch Maka from the + desktop launcher entry. +4. Configure a model connection, send one prompt, and run one representative file-tool task. +5. Run one terminal task and confirm packaged `node-pty` behavior. +6. Confirm the documented Computer Use limitation remains accurate: Computer Use is not offered on + Linux. +7. Add a clean remote Runtime Host from the packaged Desktop app. Confirm setup installs the exact + public `maka-agent@` package and the remote session completes one model turn. + Immediately before approving the `product-release` Environment, reverify that the approved ASF candidate tag and convenience `v` tag still resolve to the same recorded commit. Approve -only after npm verification and both independent-machine acceptance passes. If any required artifact, npm step, or +only after npm verification and every independent-machine acceptance pass. If any required artifact, npm step, or acceptance step fails, keep the Draft unpublished, fix the issue, increment the root product version, and run the full workflow again. Never replace an existing release identity. diff --git a/.github/workflows/desktop-nightly.yml b/.github/workflows/desktop-nightly.yml index fb03a3b485..9616f9d957 100644 --- a/.github/workflows/desktop-nightly.yml +++ b/.github/workflows/desktop-nightly.yml @@ -87,9 +87,20 @@ jobs: matrix: include: - platform: macos + arch: arm64 runner: macos-15 + - platform: macos + arch: x64 + runner: macos-15-intel - platform: windows + arch: x64 runner: windows-2025 + - platform: linux + arch: x64 + runner: ubuntu-24.04 + - platform: linux + arch: arm64 + runner: ubuntu-24.04-arm runs-on: ${{ matrix.runner }} environment: nightly timeout-minutes: 75 @@ -145,7 +156,7 @@ jobs: APPLE_API_KEY: ${{ runner.temp }}/AuthKey_Maka.p8 APPLE_API_KEY_ID: ${{ secrets.APPLE_API_KEY_ID }} APPLE_API_ISSUER: ${{ secrets.APPLE_API_ISSUER }} - run: npm run package:macos-arm64 + run: npm run package:macos-${{ matrix.arch }} - name: Notarize and verify the macOS Nightly if: matrix.platform == 'macos' @@ -153,15 +164,23 @@ jobs: APPLE_API_KEY: ${{ runner.temp }}/AuthKey_Maka.p8 APPLE_API_KEY_ID: ${{ secrets.APPLE_API_KEY_ID }} APPLE_API_ISSUER: ${{ secrets.APPLE_API_ISSUER }} - DMG_PATH: apps/desktop/release/Maka-${{ needs.identity.outputs.version }}-mac-arm64.dmg run: | + # Discovered, not spelled out — see the same step in release.yml. + shopt -s nullglob + dmgs=(apps/desktop/release/*.dmg) + if [[ ${#dmgs[@]} -ne 1 ]]; then + echo "Expected exactly one DMG to notarize, found ${#dmgs[@]}" >&2 + exit 1 + fi + DMG_PATH="${dmgs[0]}" + xcrun notarytool submit "$DMG_PATH" \ --key "$APPLE_API_KEY" \ --key-id "$APPLE_API_KEY_ID" \ --issuer "$APPLE_API_ISSUER" \ --wait xcrun stapler staple "$DMG_PATH" - npm run verify:macos-arm64 -- "$DMG_PATH" + npm run verify:macos -- "${{ matrix.arch }}" - name: Package the Windows Nightly if: matrix.platform == 'windows' @@ -169,36 +188,37 @@ jobs: - name: Verify the Windows Nightly if: matrix.platform == 'windows' - run: >- - npm run verify:windows-x64 -- - "apps/desktop/release/Maka-${{ needs.identity.outputs.version }}-win-x64.exe" + run: npm run verify:windows-x64 -- "${{ matrix.arch }}" + + - name: Package the Linux Nightly + if: matrix.platform == 'linux' + run: npm run package:linux + + - name: Ensure xvfb + if: matrix.platform == 'linux' + run: command -v xvfb-run >/dev/null 2>&1 || { sudo apt-get update && sudo apt-get install -y xvfb; } + + # `xvfb-run` because the verification launches the extracted AppImage's + # renderer over CDP, the way the macOS and Windows verifications do. + - name: Verify the Linux Nightly + if: matrix.platform == 'linux' + run: xvfb-run -a npm run verify:linux -- "${{ matrix.arch }}" - name: Stage the exact Nightly artifacts env: NIGHTLY_VERSION: ${{ needs.identity.outputs.version }} STAGE_DIRECTORY: ${{ runner.temp }}/desktop-nightly run: | - mkdir -p "$STAGE_DIRECTORY" - if [[ "${{ matrix.platform }}" = macos ]]; then - cp -- \ - "apps/desktop/release/Maka-$NIGHTLY_VERSION-mac-arm64.dmg" \ - "apps/desktop/release/Maka-$NIGHTLY_VERSION-mac-arm64.zip" \ - "apps/desktop/release/Maka-$NIGHTLY_VERSION-mac-arm64.zip.blockmap" \ - apps/desktop/release/dev-mac.yml \ - "$STAGE_DIRECTORY/" - else - cp -- \ - "apps/desktop/release/Maka-$NIGHTLY_VERSION-win-x64.exe" \ - "apps/desktop/release/Maka-$NIGHTLY_VERSION-win-x64.exe.blockmap" \ - "apps/desktop/release/Maka-$NIGHTLY_VERSION-win-x64.zip" \ - apps/desktop/release/dev.yml \ - "$STAGE_DIRECTORY/" - fi + node scripts/desktop-nightly.mjs stage-target \ + "${{ matrix.platform }}-${{ matrix.arch }}" \ + apps/desktop/release \ + "$STAGE_DIRECTORY" \ + "$NIGHTLY_VERSION" - name: Preserve the verified Nightly artifacts uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 with: - name: desktop-nightly-${{ matrix.platform }} + name: desktop-nightly-${{ matrix.platform }}-${{ matrix.arch }} path: ${{ runner.temp }}/desktop-nightly if-no-files-found: error compression-level: 0 @@ -236,7 +256,7 @@ jobs: - name: Install publisher dependencies run: npm ci --ignore-scripts - - name: Download both verified Desktop builds + - name: Download every verified Desktop build uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: pattern: desktop-nightly-* @@ -263,7 +283,15 @@ jobs: ATTESTATION_BUNDLE: ${{ steps.attest.outputs.bundle-path }} CERTIFICATE_IDENTITY: https://github.com/${{ github.repository }}/.github/workflows/desktop-nightly.yml@refs/heads/main GH_TOKEN: ${{ github.token }} + NIGHTLY_VERSION: ${{ needs.identity.outputs.version }} run: | + # Verify what is on disk, and prove the count matches what the target + # descriptor says was attested. The list is written to a file first + # because a process substitution hides its exit code, which would let + # a failing lister leave this step green with nothing verified. + node scripts/desktop-nightly.mjs attested-assets "$NIGHTLY_VERSION" \ + > "$RUNNER_TEMP/attested-assets.txt" + expected="$(wc -l < "$RUNNER_TEMP/attested-assets.txt")" verified=0 while IFS= read -r -d '' artifact; do gh attestation verify "$artifact" \ @@ -273,8 +301,8 @@ jobs: --cert-oidc-issuer https://token.actions.githubusercontent.com verified=$((verified + 1)) done < <(find "$GITHUB_WORKSPACE/.nightly-stage/release" -maxdepth 1 -type f -print0) - if (( verified != 8 )); then - echo "Expected 8 verified Desktop Nightly subjects, found $verified" >&2 + if (( verified != expected )); then + echo "Expected $expected verified Desktop Nightly subjects, found $verified" >&2 exit 1 fi diff --git a/.github/workflows/release-cli-finalize.yml b/.github/workflows/release-cli-finalize.yml index 49c7667a1d..f730d58b04 100644 --- a/.github/workflows/release-cli-finalize.yml +++ b/.github/workflows/release-cli-finalize.yml @@ -235,9 +235,8 @@ jobs: - name: Download the exact verified Release run artifacts uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 with: - pattern: release-*-${{ needs.inspect.outputs.release_run_attempt }} + name: product-release-assets-${{ needs.inspect.outputs.release_run_attempt }} path: ${{ runner.temp }}/product-release - merge-multiple: true github-token: ${{ github.token }} repository: ${{ github.repository }} run-id: ${{ needs.inspect.outputs.release_run_id }} diff --git a/.github/workflows/release-linux-check.yml b/.github/workflows/release-linux-check.yml new file mode 100644 index 0000000000..5316d3ec8e --- /dev/null +++ b/.github/workflows/release-linux-check.yml @@ -0,0 +1,131 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +name: Release Linux check + +# The same lesson the Windows lane already carries, and the reason this one +# exists: the Linux packaging path shipped with two defects that no test and no +# check could see, because nothing outside a real Linux build ever ran it. Both +# were in metadata electron-builder resolves before it packages anything — the +# executable name and the deb's homepage — and both are now also asserted in +# milliseconds by `scripts/desktop-release-targets.test.mjs`. This lane covers +# what that one structurally cannot: fpm and the AppImage runtime producing +# real payloads, the architecture of the native peer inside them, and the +# packaged application actually starting. +on: + pull_request: + # Every input whose correctness can only be observed by building on Linux. + # It reaches past the release scripts because the packaging configuration + # and the manifests own the identity fpm and the AppImage runtime demand, + # and the native peer is built per host. + paths: + - 'apps/desktop/electron-builder.config.mjs' + - 'apps/desktop/package.json' + - 'native/runtime-host-peer/**' + - 'scripts/package-linux.mjs' + - 'scripts/verify-linux.mjs' + - 'scripts/desktop-release-targets.mjs' + - 'scripts/desktop-update-contract.mjs' + - 'scripts/verify-packaged-app.mjs' + # What the two entries above import, and what they read. The packaging + # configuration pulls three scripts in, the verifier pulls in the + # third-party closure, and `assertPackagedResources` builds its required + # list by listing the icon catalogue — so adding an icon changes the + # assertion. `packages/**` is deliberately absent: the Windows lane + # already covers that half and has a closure test binding it in both + # directions, and what is unproven here is the packaging format, not the + # shared runtime. The cron below is what catches the rest. + - 'apps/desktop/assets/**' + - 'scripts/desktop-nightly.mjs' + - 'scripts/product-release-identity.mjs' + - 'scripts/release-cli-file-policy.mjs' + - 'scripts/third-party-closure.mjs' + - 'package.json' + - 'package-lock.json' + - '.github/workflows/release.yml' + - '.github/workflows/release-linux-check.yml' + # The list above is a pre-filter, not this lane's import closure, so a + # transitive edit it cannot match would otherwise first be observed on + # release day. The Nightly is not a substitute: it is a different workflow, + # reached only through a successful npm publication and a repository + # variable, so it cannot be this lane's escape from its own filter. + # `ci-workflow-policy.test.mjs` enumerates the directory for exactly this. + schedule: + - cron: '43 5 * * *' + workflow_dispatch: + +permissions: + contents: read + +# Keyed on the pull request number rather than the ref, for the reason the +# Windows lane documents: `github.ref` is refs/heads/main for the scheduled run +# and for a dispatch alike, and one shared group would let a dispatch queue +# behind the nightly and then be discarded while still pending. +concurrency: + group: release-linux-check-${{ github.event.pull_request.number || github.run_id }} + cancel-in-progress: true + +jobs: + # Not `package`: the Windows lane's job already claims that name, and two + # checks reading `package` on the same pull request cannot be told apart. + package-linux: + # x64 only. The arm64 packages are built from the same scripts, the same + # configuration and the same descriptor, so this lane would re-prove all of + # what it can reach and none of what differs — a 16K-page host and its own + # native peer. The Nightly matrix builds and verifies arm64 every night. + runs-on: ubuntu-24.04 + # The one tier every pull-request lane carries, not a per-lane value. This + # lane's first real run packaged and verified in 6m22s. + timeout-minutes: 45 + defaults: + run: + shell: bash + + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + + - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 + with: + node-version: '24' + cache: npm + + - name: Install dependencies + run: npm ci + + - name: Update stable Rust for Desktop native artifacts + run: rustup update stable --no-self-update + + # Builds the AppImage and the deb in separate electron-builder runs and + # merges their update feeds, which is the sequence the release and the + # Nightly both use. + - name: Package the Linux AppImage and deb + run: npm run package:linux + + - name: Ensure xvfb + run: command -v xvfb-run >/dev/null 2>&1 || { sudo apt-get update && sudo apt-get install -y xvfb; } + + # Opens both payloads: the AppImage must not carry the deb's + # `package-type` marker, and the deb must carry it, declare an installable + # package name and the architecture its filename claims, and satisfy the + # same resource, update and dependency assertions. The native peer's ELF + # header is read in both. Finally the extracted AppImage is launched — + # `xvfb-run` because that last step drives a real renderer over CDP, the + # way the macOS and Windows verifications already do. + - name: Verify the Linux release + run: xvfb-run -a npm run verify:linux -- x64 diff --git a/.github/workflows/release-windows-check.yml b/.github/workflows/release-windows-check.yml index 2d7b47f811..a13b76556d 100644 --- a/.github/workflows/release-windows-check.yml +++ b/.github/workflows/release-windows-check.yml @@ -142,9 +142,22 @@ jobs: run: npm run package:windows-x64 - name: Verify the Windows release + run: npm run verify:windows-x64 -- x64 + + # Discovered, not spelled out: this lane builds one installer, and the name + # is the target descriptor's to decide. The installer steps below are + # handed a path rather than a target, so unlike `verify:windows-x64` they + # cannot resolve it themselves. + - name: Locate the packaged Windows installer + id: installer run: | - version="$(node -p "require('./apps/desktop/package.json').version")" - npm run verify:windows-x64 -- "apps/desktop/release/Maka-${version}-win-x64.exe" + shopt -s nullglob + exes=(apps/desktop/release/*.exe) + if [[ ${#exes[@]} -ne 1 ]]; then + echo "Expected exactly one Windows installer, found ${#exes[@]}" >&2 + exit 1 + fi + echo "exe=${exes[0]}" >> "$GITHUB_OUTPUT" # The baseline is pinned by version, tag, asset name and SHA-256 in # `scripts/windows-upgrade-baseline.json`, so it is immutable and the @@ -174,9 +187,8 @@ jobs: # does decide the outcome. - name: Exercise pinned-version upgrade and uninstall run: | - version="$(node -p "require('./apps/desktop/package.json').version")" npm run verify:windows-installer -- \ - "apps/desktop/release/Maka-${version}-win-x64.exe" \ + "${{ steps.installer.outputs.exe }}" \ "${{ steps.previous.outputs.exe }}" - name: Build the version-bumped autoupdate installer @@ -184,9 +196,8 @@ jobs: - name: Verify automatic update end to end run: | - version="$(node -p "require('./apps/desktop/package.json').version")" npm run verify:windows-autoupdate -- \ - "apps/desktop/release/Maka-${version}-win-x64.exe" \ + "${{ steps.installer.outputs.exe }}" \ apps/desktop/release-autoupdate-next # Consumes only what this run built — the candidate installer and the @@ -194,7 +205,6 @@ jobs: # `installer.nsh` Abort path still restores the previous installation. - name: Prove deterministic mid-install failure rollback run: | - version="$(node -p "require('./apps/desktop/package.json').version")" npm run verify:windows-installer-rollback -- \ - "apps/desktop/release/Maka-${version}-win-x64.exe" \ + "${{ steps.installer.outputs.exe }}" \ apps/desktop/release-autoupdate-next diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index ee1364dae6..9039c898da 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -41,8 +41,6 @@ jobs: tag: ${{ steps.identity.outputs.tag }} source_commit: ${{ steps.identity.outputs.source_commit }} source_reference_tag: ${{ steps.identity.outputs.source_reference_tag }} - dmg: ${{ steps.identity.outputs.dmg }} - exe: ${{ steps.identity.outputs.exe }} cli_archive: ${{ steps.identity.outputs.cli_archive }} node_version: ${{ steps.identity.outputs.node_version }} npm_version: ${{ steps.identity.outputs.npm_version }} @@ -90,15 +88,28 @@ jobs: desktop: needs: release-identity strategy: - # Both platforms ship from one dispatch, so a failure on one still leaves - # the other artifact available for diagnosis. + # Every target ships from one dispatch, so a failure on one still leaves + # the other artifacts available for diagnosis. Each architecture builds on + # a runner of its own: the native Runtime Host peer and the packaged smoke + # probes are host binaries. fail-fast: false matrix: include: - platform: macos + arch: arm64 runner: macos-15 + - platform: macos + arch: x64 + runner: macos-15-intel - platform: windows + arch: x64 runner: windows-2025 + - platform: linux + arch: x64 + runner: ubuntu-24.04 + - platform: linux + arch: arm64 + runner: ubuntu-24.04-arm runs-on: ${{ matrix.runner }} environment: release timeout-minutes: 75 @@ -156,7 +167,7 @@ jobs: APPLE_API_KEY: ${{ runner.temp }}/AuthKey_Maka.p8 APPLE_API_KEY_ID: ${{ secrets.APPLE_API_KEY_ID }} APPLE_API_ISSUER: ${{ secrets.APPLE_API_ISSUER }} - run: npm run package:macos-arm64 + run: npm run package:macos-${{ matrix.arch }} - name: Notarize and staple the signed final DMG if: matrix.platform == 'macos' @@ -164,8 +175,19 @@ jobs: APPLE_API_KEY: ${{ runner.temp }}/AuthKey_Maka.p8 APPLE_API_KEY_ID: ${{ secrets.APPLE_API_KEY_ID }} APPLE_API_ISSUER: ${{ secrets.APPLE_API_ISSUER }} - DMG_PATH: apps/desktop/release/${{ needs.release-identity.outputs.dmg }} run: | + # Discovered, not spelled out: this runner builds one architecture, so + # exactly one DMG exists, and signing does not need to know its name. + # The name itself is the descriptor's to decide and `verify:macos` + # below resolves it from there. + shopt -s nullglob + dmgs=(apps/desktop/release/*.dmg) + if [[ ${#dmgs[@]} -ne 1 ]]; then + echo "Expected exactly one DMG to sign, found ${#dmgs[@]}" >&2 + exit 1 + fi + DMG_PATH="${dmgs[0]}" + codesign --verify --verbose=4 "$DMG_PATH" xcrun notarytool submit "$DMG_PATH" \ --key "$APPLE_API_KEY" \ @@ -184,21 +206,21 @@ jobs: - name: Verify the final DMG if: matrix.platform == 'macos' - run: npm run verify:macos-arm64 -- "apps/desktop/release/${{ needs.release-identity.outputs.dmg }}" + run: npm run verify:macos -- "${{ matrix.arch }}" + # The updater is one code path across both macOS architectures, so the + # end-to-end update proof runs once, on the architecture most installs + # use. package-macos-autoupdate-next.mjs enforces the same restriction. - name: Build the version-bumped macOS update - if: matrix.platform == 'macos' + if: matrix.platform == 'macos' && matrix.arch == 'arm64' env: CSC_LINK: ${{ secrets.CSC_LINK }} CSC_KEY_PASSWORD: ${{ secrets.CSC_KEY_PASSWORD }} run: npm run package:macos-autoupdate-next - name: Verify macOS automatic update end to end - if: matrix.platform == 'macos' - run: | - npm run verify:macos-autoupdate -- \ - "apps/desktop/release/Maka-${{ needs.release-identity.outputs.version }}-mac-arm64.zip" \ - apps/desktop/release-autoupdate-next + if: matrix.platform == 'macos' && matrix.arch == 'arm64' + run: npm run verify:macos-autoupdate -- apps/desktop/release-autoupdate-next # Windows has no Authenticode certificate yet, so this build is unsigned # and there is nothing to notarize between packaging and verification. @@ -208,7 +230,23 @@ jobs: - name: Verify the Windows release if: matrix.platform == 'windows' - run: npm run verify:windows-x64 -- "apps/desktop/release/${{ needs.release-identity.outputs.exe }}" + run: npm run verify:windows-x64 -- "${{ matrix.arch }}" + + # Discovered, not spelled out, for the same reason as the DMG above: this + # runner builds one installer, and the name is the descriptor's to decide. + # The installer steps below are handed a path rather than a target, so + # unlike `verify:windows-x64` they cannot resolve it themselves. + - name: Locate the packaged Windows installer + id: installer + if: matrix.platform == 'windows' + run: | + shopt -s nullglob + exes=(apps/desktop/release/*.exe) + if [[ ${#exes[@]} -ne 1 ]]; then + echo "Expected exactly one Windows installer, found ${#exes[@]}" >&2 + exit 1 + fi + echo "exe=${exes[0]}" >> "$GITHUB_OUTPUT" - name: Download and verify the pinned Windows upgrade baseline id: previous @@ -224,7 +262,7 @@ jobs: if: matrix.platform == 'windows' run: | npm run verify:windows-installer -- \ - "apps/desktop/release/${{ needs.release-identity.outputs.exe }}" \ + "${{ steps.installer.outputs.exe }}" \ "${{ steps.previous.outputs.exe }}" - name: Build the version-bumped autoupdate installer @@ -235,27 +273,43 @@ jobs: if: matrix.platform == 'windows' run: | npm run verify:windows-autoupdate -- \ - "apps/desktop/release/${{ needs.release-identity.outputs.exe }}" \ + "${{ steps.installer.outputs.exe }}" \ apps/desktop/release-autoupdate-next - name: Prove deterministic mid-install failure rollback if: matrix.platform == 'windows' run: | npm run verify:windows-installer-rollback -- \ - "apps/desktop/release/${{ needs.release-identity.outputs.exe }}" \ + "${{ steps.installer.outputs.exe }}" \ apps/desktop/release-autoupdate-next + # Linux ships unsigned for the same reason Windows does: there is no + # certificate yet, so packaging and verification are adjacent. + - name: Package the Linux AppImage and deb + if: matrix.platform == 'linux' + run: npm run package:linux + + - name: Ensure xvfb + if: matrix.platform == 'linux' + run: command -v xvfb-run >/dev/null 2>&1 || { sudo apt-get update && sudo apt-get install -y xvfb; } + + # `xvfb-run` because the verification launches the extracted AppImage's + # renderer over CDP, the way the macOS and Windows verifications do. + - name: Verify the Linux release + if: matrix.platform == 'linux' + run: xvfb-run -a npm run verify:linux -- "${{ matrix.arch }}" + - name: Stage the exact product artifact group run: >- node scripts/product-release-artifacts.mjs stage - "desktop-${{ matrix.platform }}" + "desktop-${{ matrix.platform }}-${{ matrix.arch }}" apps/desktop/release "${{ runner.temp }}/release-assets" - name: Upload the verified release assets uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 with: - name: release-desktop-${{ matrix.platform }}-${{ github.run_attempt }} + name: release-desktop-${{ matrix.platform }}-${{ matrix.arch }}-${{ github.run_attempt }} path: ${{ runner.temp }}/release-assets if-no-files-found: error retention-days: 30 @@ -401,6 +455,11 @@ jobs: - name: Install the release verifier dependencies run: npm ci --ignore-scripts --no-audit --no-fund + # Each macOS runner uploaded a feed naming only its own architecture. The + # feed a client reads carries both, so it only exists once they are here. + - name: Merge the per-architecture update feeds + run: node scripts/product-release-artifacts.mjs merge-feeds release-assets + - name: Verify the exact product artifact manifest run: node scripts/product-release-artifacts.mjs verify release-assets @@ -425,6 +484,19 @@ jobs: compression-level: 0 retention-days: 30 + # The runner uploads still carry the per-architecture feeds this job + # merged away. Finalize must publish and attest the published set, so it + # is handed these normalized bytes rather than the raw runner groups. The + # name deliberately avoids the `release-*` glob this job downloads. + - name: Upload the verified release assets + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: product-release-assets-${{ github.run_attempt }} + path: release-assets + if-no-files-found: error + compression-level: 0 + retention-days: 30 + - name: Revalidate the live ASF source reference env: SOURCE_COMMIT: ${{ needs.release-identity.outputs.source_commit }} @@ -456,9 +528,9 @@ jobs: run: | notes="Maka Desktop and the required Apple Silicon CLI/TUI artifact built from approved ASF source reference ${SOURCE_REFERENCE_TAG} at commit ${SOURCE_COMMIT}. - Before publishing, download these draft assets on another Apple Silicon Mac and on a Windows x64 machine, and complete .github/RELEASE_CHECKLIST.md. + Before publishing, download these draft assets on an Apple Silicon Mac, an Intel Mac, a Windows x64 machine, and an x64 and an arm64 Linux machine, and complete .github/RELEASE_CHECKLIST.md. - The Windows build is unsigned: SmartScreen warns on first launch, and the download has to be checked against its .sha256 file. + The Windows and Linux builds are unsigned: SmartScreen warns on first launch, no Linux package is signed either, and every download has to be checked against its .sha256 file. These files are convenience binaries built from the approved ASF source release; they are not ASF release artifacts. diff --git a/README.md b/README.md index 93d6154eb4..7399f1ac12 100644 --- a/README.md +++ b/README.md @@ -26,9 +26,9 @@

GitHub stars License: Apache 2.0 - macOS Apple Silicon + macOS Apple Silicon and Intel Windows unsigned preview - Linux not yet supported + Linux unsigned preview DeepWiki: third-party AI-generated docs 中文文档

diff --git a/README.zh-CN.md b/README.zh-CN.md index 6c64fb51ff..4ec446b5c5 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -26,9 +26,9 @@

GitHub stars License: Apache 2.0 - macOS Apple Silicon + macOS Apple Silicon 与 Intel Windows 未签名预览 - Linux 尚未支持 + Linux 未签名预览 DeepWiki：第三方 AI 生成文档 English

diff --git a/apps/desktop/electron-builder.config.mjs b/apps/desktop/electron-builder.config.mjs index 652e0cd08d..5893b1e875 100644 --- a/apps/desktop/electron-builder.config.mjs +++ b/apps/desktop/electron-builder.config.mjs @@ -215,11 +215,15 @@ const baseDesktopBuilderConfig = { to: 'licenses/renderer/SIMPLE_ICONS_LICENSE.md', }, ], + // No `target` here, or in `win`/`linux` below: electron-builder ignores the + // command line's architecture flags for any target the configuration names + // (targetFactory.computeArchToTargetNamesMap only falls back to the CLI when + // `target.arch` is absent, and returns the CLI map untouched when the CLI + // named targets). Declaring targets in both places lets them disagree, and a + // configured `arch` silently wins — which would build every architecture on + // every runner. The packaging scripts in package.json name the target and the + // architecture together and are the single authority for both. mac: { - target: [ - { target: 'dmg', arch: ['arm64'] }, - { target: 'zip', arch: ['arm64'] }, - ], category: 'public.app-category.productivity', // The bundle icon is what Finder, Launchpad and the installer show, and // none of those run our code — so it cannot follow the user's choice and @@ -257,10 +261,6 @@ const baseDesktopBuilderConfig = { writeUpdateInfo: false, }, win: { - target: [ - { target: 'nsis', arch: ['x64'] }, - { target: 'zip', arch: ['x64'] }, - ], artifactName: 'Maka-${version}-win-${arch}.${ext}', // Same reason as `mac.icon` above: the .exe, the installer and the // shortcut are drawn by the OS from this file, not by us. @@ -273,6 +273,34 @@ const baseDesktopBuilderConfig = { // electron-updater skips the check when there is none. Adding a certificate // is then the whole change — the verification follows it. }, + linux: { + // `${arch}` is not the Node architecture name here: electron-builder maps it + // through builder-util's getArtifactArchName, so x64 becomes `x86_64` for the + // AppImage and `amd64` for the deb. scripts/desktop-release-targets.mjs + // records those names and a test pins them to electron-builder's own mapping. + artifactName: 'Maka-${version}-linux-${arch}.${ext}', + // Same reason as `mac.icon` above: the launcher entry and the window + // decoration are drawn by the desktop environment from this file, not by + // the running app, so it cannot follow the user's icon choice. + icon: 'assets/app-icons/sky.png', + category: 'Development', + // Without this electron-builder names the binary after the npm package, and + // this one is scoped: `@maka/desktop` sanitizes to `@makadesktop`, which is + // not a name a desktop entry's `Exec=` can launch. Only Linux derives an + // executable name this way, which is why macOS and Windows never showed it. + executableName: 'maka', + // Electron takes its app_id — the window's WM_CLASS — from `desktopName` in + // the manifest, while the desktop entry's `StartupWMClass` falls back to the + // product name when that field is absent. `Maka` and `maka` never match, so + // the desktop environment cannot link a running window to the installed + // launcher: generic icon, and pinning it does nothing. Setting both keeps + // the entry's filename and the app_id derived from the same string. + syncDesktopName: true, + // fpm refuses to build a deb without a maintainer, and the field must + // carry an address. The project list is the only stable one; no individual + // owns the package. + maintainer: 'Apache Maka (incubating) ', + }, nsis: { // Everything stays at the one-click per-user defaults; the include only // adds the Abort-path pre-upgrade backup/rollback (and its test-only diff --git a/apps/desktop/package.json b/apps/desktop/package.json index 43f1b4b13d..4afcca17f4 100644 --- a/apps/desktop/package.json +++ b/apps/desktop/package.json @@ -5,6 +5,8 @@ "version": "0.2.0", "author": "The Maka Authors", "license": "Apache-2.0", + "homepage": "https://github.com/apache/maka", + "desktopName": "maka.desktop", "private": true, "type": "module", "main": "dist/main/main.js", @@ -30,8 +32,13 @@ "build:overlay": "node ../../scripts/build-cursor-overlay.mjs", "build:renderer": "vite build && node scripts/check-renderer-entry-output.mjs && node ../../scripts/check-third-party-notices.mjs", "build:workspace-deps": "npm --workspace @maka/core run build && npm --workspace @maka/storage run build && npm --workspace @maka/mcp run build && npm --workspace @maka/runtime run build && npm --workspace @maka/runtime-host run build && npm --workspace @maka/computer-use run build && npm --workspace @maka/ui run build", - "package:macos-arm64": "electron-builder --config electron-builder.config.mjs --mac --arm64 --publish never", - "package:windows-x64": "electron-builder --config electron-builder.config.mjs --win --x64 --publish never", + "package:macos-arm64": "electron-builder --config electron-builder.config.mjs --mac dmg zip --arm64 --publish never", + "package:macos-x64": "electron-builder --config electron-builder.config.mjs --mac dmg zip --x64 --publish never", + "package:windows-x64": "electron-builder --config electron-builder.config.mjs --win nsis zip --x64 --publish never", + "package:linux-appimage-x64": "electron-builder --config electron-builder.config.mjs --linux AppImage --x64 --publish never", + "package:linux-deb-x64": "electron-builder --config electron-builder.config.mjs --linux deb --x64 --publish never", + "package:linux-appimage-arm64": "electron-builder --config electron-builder.config.mjs --linux AppImage --arm64 --publish never", + "package:linux-deb-arm64": "electron-builder --config electron-builder.config.mjs --linux deb --arm64 --publish never", "typecheck": "tsc -p tsconfig.preload.json --noEmit && tsc -p tsconfig.main.json --noEmit && tsc -p tsconfig.renderer.json --noEmit && tsc -p tsconfig.storybook.json --noEmit", "typecheck:stories": "tsc -p tsconfig.storybook.json --noEmit", "test:dist": "node --test \"dist/main/**/*.test.js\" scripts/dev-app-runtime.test.mjs", diff --git a/apps/desktop/src/main/__tests__/app-update-attestation.test.ts b/apps/desktop/src/main/__tests__/app-update-attestation.test.ts index cda5e24984..7a548982ec 100644 --- a/apps/desktop/src/main/__tests__/app-update-attestation.test.ts +++ b/apps/desktop/src/main/__tests__/app-update-attestation.test.ts @@ -71,15 +71,9 @@ function provenanceBundle(name: string, sha256: string): Bundle { } as unknown as Bundle; } -test('download verification accepts only a trusted exact artifact subject', async (t) => { - const directory = await mkdtemp(join(tmpdir(), 'maka-update-attestation-')); - t.after(() => rm(directory, { recursive: true, force: true })); - const artifact = join(directory, 'cached-update.zip'); - const bytes = Buffer.from('attested update bytes'); - await writeFile(artifact, bytes); - const digest = createHash('sha256').update(bytes).digest('hex'); - const bundle = provenanceBundle('Maka-1.2.3-mac-arm64.zip', digest); - const bundleBytes = Buffer.from(JSON.stringify({ +function serializedBundle(name: string, sha256: string): Buffer { + const bundle = provenanceBundle(name, sha256); + return Buffer.from(JSON.stringify({ mediaType: bundle.mediaType, verificationMaterial: { certificate: { rawBytes: Buffer.from('fixture certificate').toString('base64') }, @@ -95,24 +89,162 @@ test('download verification accepts only a trusted exact artifact subject', asyn signatures: [{ sig: Buffer.from('fixture signature').toString('base64') }], }, })); +} + +const FIXTURE_VERSION = '1.2.3'; + +/** + * The payload names the release descriptor advertises, so the accept case runs + * against the artifacts a real feed offers rather than invented ones. + */ +const { desktopReleaseTargets } = (await import( + new URL('../../../../../scripts/desktop-release-targets.mjs', import.meta.url).href +)) as { + desktopReleaseTargets: ( + version: string, + options: { nightly: boolean }, + ) => { advertised: string[] }[]; +}; + +const ADVERTISED_PAYLOADS = desktopReleaseTargets(FIXTURE_VERSION, { nightly: false }).flatMap( + (target) => target.advertised, +); + +/** One cached download, named by the updater after the feed entry it chose. */ +async function stageDownload( + directory: string, + name: string, +): Promise<{ downloadedFile: string; digest: string }> { + const downloadedFile = join(directory, name); + const bytes = Buffer.from(`update bytes for ${name}`); + await writeFile(downloadedFile, bytes); + return { downloadedFile, digest: createHash('sha256').update(bytes).digest('hex') }; +} + +function feedFiles(names: readonly string[]): { url: string }[] { + return names.map((name) => ({ url: name })); +} + +/** The platform that installs an advertised payload, by the name it carries. */ +function installingPlatform(name: string): NodeJS.Platform { + if (name.includes('-mac-')) return 'darwin'; + if (name.includes('-win-')) return 'win32'; + return 'linux'; +} + +test('download verification accepts every payload the release descriptor advertises', async (t) => { + const directory = await mkdtemp(join(tmpdir(), 'maka-update-attestation-')); + t.after(() => rm(directory, { recursive: true, force: true })); + + for (const name of ADVERTISED_PAYLOADS) { + const { downloadedFile, digest } = await stageDownload(directory, name); + await verifyDownloadedUpdateAttestation({ + downloadedFile, + version: FIXTURE_VERSION, + platform: installingPlatform(name), + // A feed lists sibling payloads too; only the downloaded one is verified. + files: feedFiles(ADVERTISED_PAYLOADS), + trustRootCacheDirectory: join(directory, 'trust'), + fetchBundle: async () => serializedBundle(name, digest), + verifyBundle: async () => {}, + }); + } +}); + +test('download verification follows the chosen payload, not the running architecture', async (t) => { + const directory = await mkdtemp(join(tmpdir(), 'maka-update-rosetta-')); + t.after(() => rm(directory, { recursive: true, force: true })); + // An x64 macOS build under Rosetta reports process.arch 'x64' while + // electron-updater downloads the arm64 ZIP. + const name = `Maka-${FIXTURE_VERSION}-mac-arm64.zip`; + const { downloadedFile, digest } = await stageDownload(directory, name); + + await verifyDownloadedUpdateAttestation({ + downloadedFile, + version: FIXTURE_VERSION, + platform: 'darwin', + files: feedFiles([name, `Maka-${FIXTURE_VERSION}-mac-x64.zip`]), + trustRootCacheDirectory: join(directory, 'trust'), + fetchBundle: async () => serializedBundle(name, digest), + verifyBundle: async () => {}, + }); +}); + +test('download verification rejects a payload the feed, the version or the attestation disowns', async (t) => { + const directory = await mkdtemp(join(tmpdir(), 'maka-update-attestation-reject-')); + t.after(() => rm(directory, { recursive: true, force: true })); + const name = `Maka-${FIXTURE_VERSION}-mac-arm64.zip`; + const { downloadedFile, digest } = await stageDownload(directory, name); const options = { - downloadedFile: artifact, - version: '1.2.3', - platform: 'darwin' as const, - arch: 'arm64', + downloadedFile, + version: FIXTURE_VERSION, + platform: 'darwin' as NodeJS.Platform, + files: feedFiles([name]), trustRootCacheDirectory: join(directory, 'trust'), - fetchBundle: async () => bundleBytes, + fetchBundle: async () => serializedBundle(name, digest), verifyBundle: async () => {}, }; - await verifyDownloadedUpdateAttestation(options); await assert.rejects( - verifyDownloadedUpdateAttestation({ ...options, platform: 'win32', arch: 'x64' }), - /does not identify/u, + verifyDownloadedUpdateAttestation({ + ...options, + files: feedFiles([`Maka-${FIXTURE_VERSION}-win-x64.exe`]), + }), + /not a payload the update feed offered/u, + ); + + // A legacy feed lists no payloads at all, so nothing it served is verifiable. + await assert.rejects( + verifyDownloadedUpdateAttestation({ ...options, files: undefined }), + /not a payload the update feed offered/u, + ); + + // Same version, same extension, but not a desktop package: the CLI archive. + const cliArchive = await stageDownload(directory, `Maka-${FIXTURE_VERSION}-cli-mac-arm64.zip`); + await assert.rejects( + verifyDownloadedUpdateAttestation({ + ...options, + downloadedFile: cliArchive.downloadedFile, + files: feedFiles([`Maka-${FIXTURE_VERSION}-cli-mac-arm64.zip`]), + fetchBundle: async () => + serializedBundle(`Maka-${FIXTURE_VERSION}-cli-mac-arm64.zip`, cliArchive.digest), + }), + /not a desktop package darwin installs/u, + ); + + // Another platform's package, offered to a Windows build by a tampered feed. + const debian = await stageDownload(directory, `Maka-${FIXTURE_VERSION}-linux-amd64.deb`); + await assert.rejects( + verifyDownloadedUpdateAttestation({ + ...options, + platform: 'win32', + downloadedFile: debian.downloadedFile, + files: feedFiles([`Maka-${FIXTURE_VERSION}-linux-amd64.deb`]), + fetchBundle: async () => + serializedBundle(`Maka-${FIXTURE_VERSION}-linux-amd64.deb`, debian.digest), + }), + /not a desktop package win32 installs/u, ); - await writeFile(artifact, 'different update bytes'); - await assert.rejects(verifyDownloadedUpdateAttestation(options), /does not identify/u); + const stale = await stageDownload(directory, `Maka-1.2.2-mac-arm64.zip`); + await assert.rejects( + verifyDownloadedUpdateAttestation({ + ...options, + downloadedFile: stale.downloadedFile, + files: feedFiles([`Maka-1.2.2-mac-arm64.zip`]), + fetchBundle: async () => serializedBundle(`Maka-1.2.2-mac-arm64.zip`, stale.digest), + }), + /does not belong to version 1\.2\.3/u, + ); + + await assert.rejects( + verifyDownloadedUpdateAttestation({ + ...options, + fetchBundle: async () => + serializedBundle(name, createHash('sha256').update('other bytes').digest('hex')), + }), + /does not identify/u, + ); await assert.rejects( verifyDownloadedUpdateAttestation({ @@ -128,40 +260,21 @@ test('download verification accepts only a trusted exact artifact subject', asyn test('nightly verification fetches provenance from the versioned GitHub Release asset', async (t) => { const directory = await mkdtemp(join(tmpdir(), 'maka-nightly-attestation-')); t.after(() => rm(directory, { recursive: true, force: true })); - const artifact = join(directory, 'cached-update.zip'); - const bytes = Buffer.from('nightly update bytes'); - await writeFile(artifact, bytes); const version = '0.2.0-dev.20260829.42'; const name = `Maka-${version}-mac-arm64.zip`; - const digest = createHash('sha256').update(bytes).digest('hex'); - const bundle = provenanceBundle(name, digest); + const { downloadedFile, digest } = await stageDownload(directory, name); let fetchedUrl = ''; await verifyDownloadedUpdateAttestation({ channel: 'nightly', - downloadedFile: artifact, + downloadedFile, version, platform: 'darwin', - arch: 'arm64', + files: feedFiles([name]), trustRootCacheDirectory: join(directory, 'trust'), fetchBundle: async (url) => { fetchedUrl = url; - return Buffer.from(JSON.stringify({ - mediaType: bundle.mediaType, - verificationMaterial: { - certificate: { rawBytes: Buffer.from('fixture certificate').toString('base64') }, - tlogEntries: [], - }, - dsseEnvelope: { - payloadType: bundle.content.$case === 'dsseEnvelope' - ? bundle.content.dsseEnvelope.payloadType - : '', - payload: bundle.content.$case === 'dsseEnvelope' - ? Buffer.from(bundle.content.dsseEnvelope.payload).toString('base64') - : '', - signatures: [{ sig: Buffer.from('fixture signature').toString('base64') }], - }, - })); + return serializedBundle(name, digest); }, verifyBundle: async () => {}, }); diff --git a/apps/desktop/src/main/__tests__/app-update-service.test.ts b/apps/desktop/src/main/__tests__/app-update-service.test.ts index 5f1d3f1686..aad0ccc569 100644 --- a/apps/desktop/src/main/__tests__/app-update-service.test.ts +++ b/apps/desktop/src/main/__tests__/app-update-service.test.ts @@ -27,7 +27,10 @@ import { type AppUpdateInstallRequest, type AppUpdateStatus, } from '../app-update-service.js'; -import type { DownloadedUpdateAttestationVerifier } from '../app-update-attestation.js'; +import type { + DownloadedUpdateAttestationInput, + DownloadedUpdateAttestationVerifier, +} from '../app-update-attestation.js'; const FIRST_UPDATE_CHECK_DELAY_MS = 10_000; const UPDATE_CHECK_INTERVAL_MS = 4 * 60 * 60 * 1000; @@ -287,9 +290,13 @@ describe('AppUpdateService', () => { updater.emit('update-available', updateInfo('1.1.0')); return { isUpdateAvailable: true }; }; + const verified: DownloadedUpdateAttestationInput[] = []; const { clock, service } = createHarness({ updater, onStatusChange: (status) => statuses.push(status), + verifyDownloadedUpdate: async (input) => { + verified.push(input); + }, }); service.start(); @@ -305,10 +312,20 @@ describe('AppUpdateService', () => { }); updater.emit('update-downloaded', { ...updateInfo('1.1.0'), - downloadedFile: '/tmp/maka-update.zip', + files: [{ url: 'Maka-1.1.0-mac-arm64.zip', sha512: '', size: 1 }], + downloadedFile: '/tmp/Maka-1.1.0-mac-arm64.zip', }); await settleUpdateVerification(); + // The verifier identifies the payload the updater chose, so the event's + // own file list has to reach it alongside the cached file. + assert.deepEqual(verified, [ + { + downloadedFile: '/tmp/Maka-1.1.0-mac-arm64.zip', + version: '1.1.0', + files: [{ url: 'Maka-1.1.0-mac-arm64.zip', sha512: '', size: 1 }], + }, + ]); assert.deepEqual(statuses.map((status) => status.state), [ 'checking', 'available', diff --git a/apps/desktop/src/main/__tests__/runtime-host-setup-package.test.ts b/apps/desktop/src/main/__tests__/runtime-host-setup-package.test.ts index 432f50d214..6b3334113b 100644 --- a/apps/desktop/src/main/__tests__/runtime-host-setup-package.test.ts +++ b/apps/desktop/src/main/__tests__/runtime-host-setup-package.test.ts @@ -109,6 +109,30 @@ test('development setup lazily caches CLI archives by peer target unless overrid assert.equal(closes, 2); }); +test('a packaged desktop resolves its own setup package without a development peer target', async (t) => { + const appPath = await mkdtemp(join(tmpdir(), 'maka-runtime-host-setup-packaged-')); + t.after(() => rm(appPath, { recursive: true, force: true })); + await writeFile( + join(appPath, 'package.json'), + JSON.stringify({ runtimeHostSetupPackage: 'maka-agent@1.2.3' }), + ); + // Packaged builds ship on tuples that have no npm prebuild — macOS x64 — so + // reading the development peer target here would fail the local Runtime Host. + const resolver = createRuntimeHostSetupPackageResolver({ + isPackaged: true, + appPath, + environment: {}, + startDevelopmentArchiveBuild: () => assert.fail('a packaged desktop must not build the CLI'), + }); + t.after(() => resolver.close()); + + assert.equal(resolver.mode, 'published'); + assert.deepEqual(await resolver.resolveForThisDesktop(), { + kind: 'npm', + specifier: 'maka-agent@1.2.3', + }); +}); + test('cancelling the last waiter closes its build before a new setup starts', async (t) => { const directory = await mkdtemp(join(tmpdir(), 'maka-runtime-host-setup-package-')); t.after(() => rm(directory, { recursive: true, force: true })); diff --git a/apps/desktop/src/main/app-update-attestation.ts b/apps/desktop/src/main/app-update-attestation.ts index 1a7aa9e67a..bc8a4775ef 100644 --- a/apps/desktop/src/main/app-update-attestation.ts +++ b/apps/desktop/src/main/app-update-attestation.ts @@ -22,6 +22,7 @@ import { getTrustedRoot } from '@sigstore/tuf'; import { toSignedEntity, toTrustMaterial, Verifier } from '@sigstore/verify'; import { createHash } from 'node:crypto'; import { createReadStream } from 'node:fs'; +import { basename } from 'node:path'; const PRODUCT_REPOSITORY = 'apache/maka'; const PRODUCT_RELEASE_WORKFLOW = '.github/workflows/release-cli-finalize.yml'; @@ -42,9 +43,16 @@ type AttestationStatement = { readonly subject?: unknown; }; +/** The shape `UpdateDownloadedEvent.files` shares with every update feed entry. */ +export type DownloadedUpdateFile = { + readonly url: string; +}; + export type DownloadedUpdateAttestationInput = { readonly downloadedFile: string; readonly version: string; + /** Absent on a legacy feed that never listed its payloads. */ + readonly files?: readonly DownloadedUpdateFile[]; }; export type DownloadedUpdateAttestationVerifier = ( @@ -53,9 +61,8 @@ export type DownloadedUpdateAttestationVerifier = ( type VerifyDownloadedUpdateAttestationOptions = DownloadedUpdateAttestationInput & { readonly channel?: DesktopUpdateChannel; - readonly trustRootCacheDirectory: string; readonly platform?: NodeJS.Platform; - readonly arch?: string; + readonly trustRootCacheDirectory: string; readonly fetchBundle?: (url: string) => Promise; readonly verifyBundle?: (bundle: Bundle) => Promise; }; @@ -82,14 +89,48 @@ function productWorkflowSigner(channel: DesktopUpdateChannel): RegExp { ); } -function exactDesktopUpdateArtifactName( +function feedFileName(url: string): string { + return basename(url.split(/[?#]/u)[0] ?? ''); +} + +/** + * The desktop packages a platform installs, matched against what follows + * `Maka--`. Each entry names the platform and the package formats it + * accepts and deliberately says nothing about the architecture. + */ +const INSTALLABLE_UPDATE_PACKAGE: Partial> = { + win32: /^win-.+\.exe$/u, + darwin: /^mac-.+\.zip$/u, + linux: /^linux-.+\.(?:AppImage|deb)$/u, +}; + +/** + * The payload the updater actually chose. electron-updater names the cached + * file after the basename of the feed entry it downloaded, so the download is + * identified by that name rather than by this process' architecture — which + * does not decide it: macOS serves the arm64 ZIP to an x64 build under Rosetta. + * The platform only decides the package format, never the architecture: one + * Linux tuple serves either the AppImage or the deb depending on how the + * running copy was installed. + */ +function downloadedUpdateArtifactName( + downloadedFile: string, version: string, + files: readonly DownloadedUpdateFile[] | undefined, platform: NodeJS.Platform, - arch: string, ): string { - if (platform === 'darwin' && arch === 'arm64') return `Maka-${version}-mac-arm64.zip`; - if (platform === 'win32' && arch === 'x64') return `Maka-${version}-win-x64.exe`; - throw new Error(`Automatic updates are unsupported on ${platform}/${arch}`); + const name = basename(downloadedFile); + if (!(files ?? []).some((file) => feedFileName(file.url) === name)) { + throw new Error('Downloaded update is not a payload the update feed offered'); + } + const prefix = `Maka-${version}-`; + if (!name.startsWith(prefix)) { + throw new Error(`Downloaded update does not belong to version ${version}`); + } + if (!INSTALLABLE_UPDATE_PACKAGE[platform]?.test(name.slice(prefix.length))) { + throw new Error(`Downloaded update is not a desktop package ${platform} installs`); + } + return name; } function productReleaseAttestationName(version: string): string { @@ -210,10 +251,11 @@ export async function verifyDownloadedUpdateAttestation( options: VerifyDownloadedUpdateAttestationOptions, ): Promise { const version = options.version.trim().replace(/^v/iu, ''); - const expectedName = exactDesktopUpdateArtifactName( + const expectedName = downloadedUpdateArtifactName( + options.downloadedFile, version, + options.files, options.platform ?? process.platform, - options.arch ?? process.arch, ); const channel = options.channel ?? 'release'; const [artifactSha256, bundleBytes] = await Promise.all([ diff --git a/apps/desktop/src/main/app-update-service.ts b/apps/desktop/src/main/app-update-service.ts index 2ae2e4fabc..155bb22dc0 100644 --- a/apps/desktop/src/main/app-update-service.ts +++ b/apps/desktop/src/main/app-update-service.ts @@ -336,6 +336,7 @@ export function createAppUpdateService(deps: AppUpdateServiceDeps): AppUpdateSer deps.verifyDownloadedUpdate({ downloadedFile: event.downloadedFile, version, + files: event.files, }), ); activeVerification = verification; diff --git a/apps/desktop/src/main/runtime-host-boot.ts b/apps/desktop/src/main/runtime-host-boot.ts index 3d352e8370..fc86ad692a 100644 --- a/apps/desktop/src/main/runtime-host-boot.ts +++ b/apps/desktop/src/main/runtime-host-boot.ts @@ -197,7 +197,6 @@ import { } from './runtime-host-wsl-controller.js'; import { createRuntimeHostSetupPackageResolver, - desktopRuntimeHostDevelopmentPeerTarget, } from "./runtime-host-setup-package.js"; import { configureDesktopRuntimeHostPeerClient, @@ -456,10 +455,7 @@ const localRuntimeHostRemoteAccess = createDesktopLocalRuntimeHostRemoteAccess({ rootId: startupLocalStorageRoot.rootId, directPeerAvailable: runtimeHostDirectPeerAvailable, manager: () => runtimeHostManager, - resolveSetupPackage: (signal) => runtimeHostSetupPackage.resolve( - desktopRuntimeHostDevelopmentPeerTarget(), - signal, - ), + resolveSetupPackage: (signal) => runtimeHostSetupPackage.resolveForThisDesktop(signal), operator: localRuntimeHostOperator, }); const native = assembleDesktopNativeCapabilities({ @@ -566,9 +562,7 @@ const localRuntimeHostManagement = createDesktopRuntimeHostLocalManagement({ remoteAccess: localRuntimeHostRemoteAccess, operator: localRuntimeHostOperator, rootPath: startupLocalStorageRoot.canonicalPath, - resolveUpdatePackage: () => runtimeHostSetupPackage.resolve( - desktopRuntimeHostDevelopmentPeerTarget(), - ), + resolveUpdatePackage: () => runtimeHostSetupPackage.resolveForThisDesktop(), currentHostEpoch: () => runtimeHostManager?.current('local')?.candidate?.client.hostEpoch, awaitUpdatedConnection: async (previousHostEpoch, replacementExpected) => { @@ -810,11 +804,12 @@ const updateService = createAppUpdateService({ // the Sigstore verifier below. verifyDownloadedUpdate: updateTestFeed ? async () => {} - : ({ downloadedFile, version }) => + : ({ downloadedFile, version, files }) => verifyDownloadedUpdateAttestation({ channel: desktopUpdateChannel, downloadedFile, version, + files, trustRootCacheDirectory: join(userDataDir, "update-trust", "sigstore"), }), prepareInstall: async (input) => { diff --git a/apps/desktop/src/main/runtime-host-setup-package.ts b/apps/desktop/src/main/runtime-host-setup-package.ts index adfd04a85a..697ee54747 100644 --- a/apps/desktop/src/main/runtime-host-setup-package.ts +++ b/apps/desktop/src/main/runtime-host-setup-package.ts @@ -75,14 +75,22 @@ export type DesktopRuntimeHostDevelopmentPeerTarget = export interface RuntimeHostSetupPackageResolver { readonly mode: 'published' | 'development'; + /** Resolve for a peer target the caller chose, such as a probed SSH host. */ resolve( peerTarget: DesktopRuntimeHostDevelopmentPeerTarget, signal?: AbortSignal, ): Promise; + /** + * Resolve for the desktop's own machine. The development peer target is read + * only where a development build needs it to pick an npm prebuild, so a + * packaged build on a tuple that has no prebuild — macOS x64 — still resolves + * the setup package it ships with. + */ + resolveForThisDesktop(signal?: AbortSignal): Promise; close(): Promise; } -export function desktopRuntimeHostDevelopmentPeerTarget( +function desktopRuntimeHostDevelopmentPeerTarget( platform: NodeJS.Platform = process.platform, arch: string = process.arch, ): Exclude { @@ -174,29 +182,37 @@ export function createRuntimeHostSetupPackageResolver(input: { } }; - return { - mode: input.isPackaged ? 'published' : 'development', - async resolve(peerTarget, signal) { - if (closed) throw new Error('Runtime Host setup package resolver is closed'); - if (input.isPackaged) return packagedSetupPackage(input.appPath); + const resolveSetupPackage = async ( + readPeerTarget: () => DesktopRuntimeHostDevelopmentPeerTarget, + signal?: AbortSignal, + ): Promise => { + if (closed) throw new Error('Runtime Host setup package resolver is closed'); + if (input.isPackaged) return packagedSetupPackage(input.appPath); - const override = input.environment[DEVELOPMENT_ARCHIVE_ENV]; - if (override) { - const snapshot = await waitForPackage(resolveOverrideSnapshot(override), signal); - return snapshot.setupPackage; - } + const override = input.environment[DEVELOPMENT_ARCHIVE_ENV]; + if (override) { + const snapshot = await waitForPackage(resolveOverrideSnapshot(override), signal); + return snapshot.setupPackage; + } - const build = await acquireBuild(peerTarget, signal); - build.waiters += 1; - try { - return await waitForPackage(build.result, signal); - } finally { - build.waiters -= 1; - if (signal?.aborted && build.waiters === 0 && !build.settled) { - await stopBuild(peerTarget, build); - } + const peerTarget = readPeerTarget(); + const build = await acquireBuild(peerTarget, signal); + build.waiters += 1; + try { + return await waitForPackage(build.result, signal); + } finally { + build.waiters -= 1; + if (signal?.aborted && build.waiters === 0 && !build.settled) { + await stopBuild(peerTarget, build); } - }, + } + }; + + return { + mode: input.isPackaged ? 'published' : 'development', + resolve: (peerTarget, signal) => resolveSetupPackage(() => peerTarget, signal), + resolveForThisDesktop: (signal) => + resolveSetupPackage(desktopRuntimeHostDevelopmentPeerTarget, signal), async close() { if (closed) return; closed = true; diff --git a/docs/cli-distribution.md b/docs/cli-distribution.md index af6e49bc90..9f2655ed05 100644 --- a/docs/cli-distribution.md +++ b/docs/cli-distribution.md @@ -90,7 +90,7 @@ consume the standalone ZIP. | --- | --- | --- | | Which file owns the product version? | Root `package.json`; Desktop and CLI manifests must match it. | `product-release-identity.mjs` and release contract tests | | Which event defines the Apache release? | The approved source archive and vote result. The `v` tag and Draft GitHub Release identify convenience distributions built from that source commit. | ASF source-release workflow plus `release.yml` identity and exact-tag checks | -| Which convenience artifacts are required? | macOS and Windows Desktop installers and update assets plus the macOS arm64 standalone CLI ZIP. | The exact manifest from `product-release-identity.mjs`, enforced by each artifact job and the publish job | +| Which convenience artifacts are required? | macOS, Windows, and Linux Desktop installers and update assets plus the macOS arm64 standalone CLI ZIP. | The exact manifest from `product-release-identity.mjs`, enforced by each artifact job and the publish job | | Is npm another release authority? | No. It is an optional install channel whose Stage ref, source, workflow identity, and provenance all resolve to the existing product tag commit. | Tag-dispatched OIDC staging and read-only finalization; no npm-specific tag or GitHub Release | | Does the standalone CLI define another package policy? | No. It derives the workspace closure, third-party pruning, notices, and Eval runtime assets from their current manifests and shared policy. | Packaging and artifact contract tests | | Which commands are public? | `maka` only; TUI is its default mode. | CLI manifest, help tests, wrapper, and release metadata | diff --git a/docs/cli-npm-release.md b/docs/cli-npm-release.md index 0b5fc3f60d..51c7a7ce1c 100644 --- a/docs/cli-npm-release.md +++ b/docs/cli-npm-release.md @@ -132,7 +132,8 @@ The two workflows publish in this order: 2. publish the exact npm tarball with provenance under `nightly`; 3. require both the exact version and `nightly` tag to be readable from the public registry; 4. build, verify, and attest the exact Desktop packages and GitHub `dev` metadata; -5. bind a protected `v` tag to the exact source commit and verify all nine draft assets; +5. bind a protected `v` tag to the exact source commit and verify exactly the draft assets that + `desktopNightlyReleaseAssetNames` defines; 6. publish the GitHub prerelease with Latest disabled only after the draft is complete. This ordering prevents Desktop from advertising a Runtime Host version that npm does not have and diff --git a/docs/cli-npm-release.zh-CN.md b/docs/cli-npm-release.zh-CN.md index 989a59f04d..93b22e552a 100644 --- a/docs/cli-npm-release.zh-CN.md +++ b/docs/cli-npm-release.zh-CN.md @@ -117,7 +117,8 @@ authentication and disallow tokens**，然后撤销不再使用的 publish token 2. 使用 provenance 将精确 npm tarball 发布到 `nightly`； 3. 要求公共 registry 中的精确版本和 `nightly` tag 都已可读； 4. 构建、验证并 attest 精确的 Desktop 安装包和 GitHub `dev` metadata； -5. 将受保护的 `v` tag 绑定到精确 source commit，并验证 Draft 中全部九项资产； +5. 将受保护的 `v` tag 绑定到精确 source commit，并验证 Draft 中恰好是 + `desktopNightlyReleaseAssetNames` 定义的那些资产； 6. 仅在 Draft 完整后发布 Latest 关闭的 GitHub prerelease。 这个顺序既避免 Desktop 指向 npm 中不存在的 Runtime Host，也让 npm Nightly 与 Desktop 打包彼此 diff --git a/docs/computer-use-provenance.md b/docs/computer-use-provenance.md index 50e0d02b8f..0c5f56386b 100644 --- a/docs/computer-use-provenance.md +++ b/docs/computer-use-provenance.md @@ -56,7 +56,7 @@ to travel with it. Maka has one Computer Use executor, `maka-cu`. It is built from Maka's own source by the `prepare` command in `scripts/computer-use.mjs` and pinned by digest in `apps/desktop/bundled-tools.json`. It is not signed, so it is not distributed: -its `distributionReady` is false and `scripts/verify-macos-arm64-dmg.mjs` +its `distributionReady` is false and `scripts/verify-macos-dmg.mjs` forbids its path in a packaged build. Nothing third-party ships inside the Maka artifact for Computer Use, which is why the table above has one row. diff --git a/package.json b/package.json index 08a3131b68..a7cbdbc864 100644 --- a/package.json +++ b/package.json @@ -74,9 +74,10 @@ "check:runtime-host-peer-dependencies": "node scripts/generate-runtime-host-peer-dependencies.mjs --check", "generate:runtime-host-peer-notices": "node scripts/generate-runtime-host-peer-notices.mjs", "check:runtime-host-peer-notices": "node scripts/generate-runtime-host-peer-notices.mjs --check", - "check:release": "npm run check:stale && npm run check:third-party-notices && npm run check:cli-third-party-notices && npm run check:model-metadata && npm run check:product-release-identity && npm run check:asf-npm && node --test scripts/product-nightly.test.mjs scripts/desktop-nightly.test.mjs scripts/desktop-nightly-stage.test.mjs scripts/desktop-nightly-release.test.mjs scripts/desktop-nightly-workflow-policy.test.mjs scripts/product-release.test.mjs scripts/product-release-authority.test.mjs scripts/release-cli-file-policy.test.mjs scripts/release-cli-artifact-policy.test.mjs scripts/release-cli-eval-support.test.mjs scripts/release-cli-publication.test.mjs scripts/release-cli-runtime-host-diagnostics.test.mjs scripts/qualify-released-cli-state-root.test.mjs scripts/release-cli-workflow-policy.test.mjs scripts/verify-packaged-app.test.mjs scripts/third-party-closure.test.mjs scripts/generate-third-party-notices.test.mjs scripts/source-legal-inventory.test.mjs scripts/sync-model-metadata.test.mjs scripts/prepare-windows-upgrade-baseline.test.mjs scripts/windows-package-source-closure.test.mjs", - "package:macos-arm64": "node scripts/package-macos-arm64.mjs", - "verify:macos-arm64": "node scripts/verify-macos-arm64-dmg.mjs", + "check:release": "npm run check:stale && npm run check:third-party-notices && npm run check:cli-third-party-notices && npm run check:model-metadata && npm run check:product-release-identity && npm run check:asf-npm && node --test scripts/product-nightly.test.mjs scripts/desktop-release-targets.test.mjs scripts/verify-linux-harness.test.mjs scripts/desktop-nightly.test.mjs scripts/desktop-nightly-stage.test.mjs scripts/desktop-nightly-release.test.mjs scripts/desktop-nightly-workflow-policy.test.mjs scripts/product-release.test.mjs scripts/product-release-authority.test.mjs scripts/release-cli-file-policy.test.mjs scripts/release-cli-artifact-policy.test.mjs scripts/release-cli-eval-support.test.mjs scripts/release-cli-publication.test.mjs scripts/release-cli-runtime-host-diagnostics.test.mjs scripts/qualify-released-cli-state-root.test.mjs scripts/release-cli-workflow-policy.test.mjs scripts/verify-packaged-app.test.mjs scripts/third-party-closure.test.mjs scripts/generate-third-party-notices.test.mjs scripts/source-legal-inventory.test.mjs scripts/sync-model-metadata.test.mjs scripts/prepare-windows-upgrade-baseline.test.mjs scripts/windows-package-source-closure.test.mjs", + "package:macos-arm64": "node scripts/package-macos.mjs arm64", + "package:macos-x64": "node scripts/package-macos.mjs x64", + "verify:macos": "node scripts/verify-macos-dmg.mjs", "package:macos-autoupdate-next": "node scripts/package-macos-autoupdate-next.mjs", "verify:macos-autoupdate": "node scripts/verify-macos-autoupdate.mjs", "package:windows-x64": "node scripts/package-windows-x64.mjs", @@ -85,6 +86,8 @@ "package:windows-autoupdate-next": "node scripts/package-windows-autoupdate-next.mjs", "verify:windows-autoupdate": "node scripts/verify-windows-autoupdate.mjs", "verify:windows-installer-rollback": "node scripts/verify-windows-installer-rollback.mjs", + "package:linux": "node scripts/package-linux.mjs", + "verify:linux": "node scripts/verify-linux.mjs", "astryx:theme": "node scripts/build-astryx-theme.mjs", "astryx:surface-inventory": "node scripts/check-astryx-surface-inventory.mjs", "astryx:surface-inventory:write": "node scripts/generate-astryx-surface-inventory.mjs", diff --git a/scripts/ci-test-plan.mjs b/scripts/ci-test-plan.mjs index 149dd30152..5211bdf344 100644 --- a/scripts/ci-test-plan.mjs +++ b/scripts/ci-test-plan.mjs @@ -48,11 +48,13 @@ const RELEASE_CONTRACT_FILES = new Set([ '.github/workflows/release-cli-finalize.yml', '.github/workflows/release-cli-stage.yml', '.github/workflows/release.yml', + '.github/workflows/release-linux-check.yml', '.github/workflows/release-windows-check.yml', '.github/workflows/windows-recovery.yml', - 'scripts/package-macos-arm64.mjs', + 'scripts/package-macos.mjs', 'scripts/package-macos-autoupdate-next.mjs', 'scripts/package-macos-arm64-cli.mjs', + 'scripts/package-linux.mjs', 'scripts/package-windows-autoupdate-next.mjs', 'scripts/package-windows-x64.mjs', 'scripts/prepare-windows-upgrade-baseline.mjs', @@ -64,8 +66,12 @@ const RELEASE_CONTRACT_FILES = new Set([ 'scripts/release-version.mjs', 'scripts/third-party-closure.test.mjs', 'scripts/verify-macos-arm64-cli.mjs', - 'scripts/verify-macos-arm64-dmg.mjs', + 'scripts/verify-macos-dmg.mjs', 'scripts/verify-macos-autoupdate.mjs', + 'scripts/verify-linux.mjs', + 'scripts/verify-linux-harness.test.mjs', + 'scripts/desktop-release-targets.mjs', + 'scripts/desktop-release-targets.test.mjs', 'scripts/desktop-update-contract.mjs', 'scripts/product-nightly.mjs', 'scripts/product-nightly.test.mjs', diff --git a/scripts/ci-test-plan.test.mjs b/scripts/ci-test-plan.test.mjs index a9d03ddeb0..cbd19b7100 100644 --- a/scripts/ci-test-plan.test.mjs +++ b/scripts/ci-test-plan.test.mjs @@ -219,9 +219,10 @@ test('release authority changes select their dedicated contract gate', () => { '.github/workflows/release-cli-finalize.yml', '.github/workflows/release-cli-stage.yml', '.github/workflows/release.yml', - 'scripts/package-macos-arm64.mjs', + 'scripts/package-macos.mjs', 'scripts/package-macos-autoupdate-next.mjs', 'scripts/package-macos-arm64-cli.mjs', + 'scripts/package-linux.mjs', 'scripts/package-windows-x64.mjs', 'scripts/prepare-windows-upgrade-baseline.mjs', 'scripts/product-release-artifacts.mjs', @@ -234,8 +235,11 @@ test('release authority changes select their dedicated contract gate', () => { 'scripts/release-version.mjs', 'scripts/release-cli-publication.test.mjs', 'scripts/verify-macos-arm64-cli.mjs', - 'scripts/verify-macos-arm64-dmg.mjs', + 'scripts/verify-macos-dmg.mjs', 'scripts/verify-macos-autoupdate.mjs', + 'scripts/verify-linux.mjs', + 'scripts/desktop-release-targets.mjs', + 'scripts/desktop-release-targets.test.mjs', 'scripts/desktop-update-contract.mjs', 'scripts/verify-packaged-app.mjs', 'scripts/verify-windows-x64.mjs', diff --git a/scripts/ci-workflow-policy.test.mjs b/scripts/ci-workflow-policy.test.mjs index d0cd004fb4..2175d425d5 100644 --- a/scripts/ci-workflow-policy.test.mjs +++ b/scripts/ci-workflow-policy.test.mjs @@ -307,6 +307,7 @@ test('pull request triggers stay on an explicit allowlist', () => { 'dependency-audit.yml', 'gitoxide-helper-admission.yml', 'pr-effort-label.yml', + 'release-linux-check.yml', 'release-windows-check.yml', 'runtime-host-owner-platform.yml', 'runtime-host-peer-admission.yml', @@ -512,6 +513,16 @@ test('the packaged Windows gate triggers on release orchestration changes', () = assert.match(workflow, /'\.github\/workflows\/release\.yml'/u); }); +test('the packaged Windows gate never spells the installer it built', () => { + // `scripts/desktop-release-targets.mjs` names every distributable. This lane + // hands the architecture to `verify:windows-x64`, which reads the descriptor, + // and discovers the one packaged `.exe` for the steps that need a path. + const workflow = readWorkflow('release-windows-check.yml'); + + assert.doesNotMatch(workflow, /win-x64\.exe/u); + assert.match(workflow, /exes=\(apps\/desktop\/release\/\*\.exe\)/u); +}); + test('the packaged Windows gate workflow is itself a release-contract input', () => { assert.equal(planTests(['.github/workflows/release-windows-check.yml']).releaseContract, true); assert.match( @@ -533,6 +544,21 @@ test('the packaged Windows gate triggers on the worker copy step it cannot impor ); }); +test('the packaged Linux gate verifies under a virtual display', () => { + // The last thing `verify:linux` does is launch the extracted AppImage's + // renderer over CDP. A headless runner has no display, so a step that dropped + // `xvfb-run` would fail the lane at its slowest point. + const runs = + readWorkflow('release-linux-check.yml') + .replaceAll(/^[ \t]*#.*$/gmu, '') + .match(/^[ \t]*run: .*npm run verify:linux.*$/gmu) ?? []; + + assert.equal(runs.length, 1); + for (const run of runs) { + assert.match(run, /run: xvfb-run\b/u, run); + } +}); + test('pull-request and release lanes share the packaged sandbox lifecycle verifier', () => { for (const name of ['release-windows-check.yml', 'release.yml']) { assert.match(readWorkflow(name), /npm run verify:windows-x64/u, name); diff --git a/scripts/desktop-nightly-fixture.mjs b/scripts/desktop-nightly-fixture.mjs new file mode 100644 index 0000000000..ca8f355466 --- /dev/null +++ b/scripts/desktop-nightly-fixture.mjs @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import { createHash } from 'node:crypto'; +import { writeFile } from 'node:fs/promises'; +import { join } from 'node:path'; +import { stringify } from 'yaml'; +import { desktopReleaseTargets } from './desktop-release-targets.mjs'; + +/** + * Builds what the packaging runners would have uploaded, so a test can exercise + * staging and publication without producing real installers. The descriptor + * names every file, blockmaps included, so this stays in step with it. Both + * channels stage the same shape, so both are built here; a formal release adds + * checksum sidecars, which only its own caller knows the contract for. + */ +export async function writeDesktopReleaseInput(directory, version, { nightly = false } = {}) { + await Promise.all( + desktopReleaseTargets(version, { nightly }).map(async (target) => { + const files = []; + for (const artifact of target.payloads) { + const bytes = Buffer.from(`${artifact} bytes`); + await writeFile(join(directory, artifact), bytes); + if (!target.advertised.includes(artifact)) continue; + files.push({ + url: artifact, + sha512: createHash('sha512').update(bytes).digest('base64'), + size: bytes.byteLength, + }); + } + await writeFile( + join(directory, target.feed), + stringify({ + version, + files, + path: files[0].url, + sha512: files[0].sha512, + releaseDate: '2026-08-29T18:17:00.000Z', + }), + ); + }), + ); +} diff --git a/scripts/desktop-nightly-release.test.mjs b/scripts/desktop-nightly-release.test.mjs index 00d0ce8e49..c415b54447 100644 --- a/scripts/desktop-nightly-release.test.mjs +++ b/scripts/desktop-nightly-release.test.mjs @@ -32,34 +32,14 @@ import { desktopNightlyReleaseAssetNames, stageDesktopNightly, } from './desktop-nightly.mjs'; +import { writeDesktopReleaseInput } from './desktop-nightly-fixture.mjs'; import { productReleaseArtifactRecords } from './product-release-artifacts.mjs'; -async function writeUpdateSet(directory, version, platform) { - const isMac = platform === 'mac'; - const artifact = isMac ? `Maka-${version}-mac-arm64.zip` : `Maka-${version}-win-x64.exe`; - const metadata = isMac ? 'dev-mac.yml' : 'dev.yml'; - const bytes = Buffer.from(`${platform} nightly bytes`); - const sha512 = createHash('sha512').update(bytes).digest('base64'); - await Promise.all([ - writeFile(join(directory, artifact), bytes), - writeFile(join(directory, `${artifact}.blockmap`), `${platform} blockmap`), - writeFile( - join(directory, metadata), - `version: ${version}\nfiles:\n - url: ${artifact}\n sha512: ${sha512}\n size: ${bytes.length}\npath: ${artifact}\nsha512: ${sha512}\n`, - ), - ]); -} - async function stageRelease(root, version) { const input = join(root, 'input'); const output = join(root, 'output'); await mkdir(input); - await Promise.all([ - writeUpdateSet(input, version, 'mac'), - writeUpdateSet(input, version, 'win'), - writeFile(join(input, `Maka-${version}-mac-arm64.dmg`), 'dmg'), - writeFile(join(input, `Maka-${version}-win-x64.zip`), 'windows zip'), - ]); + await writeDesktopReleaseInput(input, version, { nightly: true }); await stageDesktopNightly({ inputDirectory: input, outputDirectory: output, diff --git a/scripts/desktop-nightly-stage.test.mjs b/scripts/desktop-nightly-stage.test.mjs index c8a23a5388..c249fa1b53 100644 --- a/scripts/desktop-nightly-stage.test.mjs +++ b/scripts/desktop-nightly-stage.test.mjs @@ -23,30 +23,11 @@ import { mkdir, mkdtemp, readFile, readdir, rm, writeFile } from 'node:fs/promis import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { test } from 'node:test'; -import { stringify } from 'yaml'; +import { parse, stringify } from 'yaml'; +import { writeDesktopReleaseInput } from './desktop-nightly-fixture.mjs'; import { addDesktopNightlyAttestation, stageDesktopNightly } from './desktop-nightly.mjs'; import { verifyDesktopUpdateArtifacts } from './desktop-update-contract.mjs'; -async function writeUpdateSet(directory, version, platform) { - const isMac = platform === 'mac'; - const artifact = isMac ? `Maka-${version}-mac-arm64.zip` : `Maka-${version}-win-x64.exe`; - const metadata = isMac ? 'dev-mac.yml' : 'dev.yml'; - const bytes = Buffer.from(`${platform} nightly bytes`); - const sha512 = createHash('sha512').update(bytes).digest('base64'); - await writeFile(join(directory, artifact), bytes); - await writeFile(join(directory, `${artifact}.blockmap`), `${platform} blockmap`); - await writeFile( - join(directory, metadata), - stringify({ - version, - files: [{ url: artifact, sha512, size: bytes.byteLength }], - path: artifact, - sha512, - releaseDate: '2026-08-29T18:17:00.000Z', - }), - ); -} - test('staging creates only the exact GitHub Release assets', async (t) => { const root = await mkdtemp(join(tmpdir(), 'maka-desktop-nightly-')); t.after(() => rm(root, { recursive: true, force: true })); @@ -54,12 +35,7 @@ test('staging creates only the exact GitHub Release assets', async (t) => { const output = join(root, 'output'); const version = '0.2.0-dev.42.20260829'; await mkdir(input); - await Promise.all([ - writeUpdateSet(input, version, 'mac'), - writeUpdateSet(input, version, 'win'), - writeFile(join(input, `Maka-${version}-mac-arm64.dmg`), 'dmg'), - writeFile(join(input, `Maka-${version}-win-x64.zip`), 'windows zip'), - ]); + await writeDesktopReleaseInput(input, version, { nightly: true }); await stageDesktopNightly({ inputDirectory: input, @@ -71,9 +47,18 @@ test('staging creates only the exact GitHub Release assets', async (t) => { `Maka-${version}-mac-arm64.dmg`, `Maka-${version}-mac-arm64.zip`, `Maka-${version}-mac-arm64.zip.blockmap`, + `Maka-${version}-mac-x64.dmg`, + `Maka-${version}-mac-x64.zip`, + `Maka-${version}-mac-x64.zip.blockmap`, `Maka-${version}-win-x64.exe`, `Maka-${version}-win-x64.exe.blockmap`, `Maka-${version}-win-x64.zip`, + // Linux artifact names use the packaging ecosystem's architecture, and + // neither distributable has a sidecar blockmap. + `Maka-${version}-linux-x86_64.AppImage`, + `Maka-${version}-linux-amd64.deb`, + `Maka-${version}-linux-arm64.AppImage`, + `Maka-${version}-linux-arm64.deb`, ]; const release = join(output, 'release'); for (const name of payloadNames) { @@ -84,19 +69,39 @@ test('staging creates only the exact GitHub Release assets', async (t) => { directory: release, metadataName: 'dev-mac.yml', version, - artifactName: `Maka-${version}-mac-arm64.zip`, + artifactNames: [`Maka-${version}-mac-arm64.zip`, `Maka-${version}-mac-x64.zip`], }), verifyDesktopUpdateArtifacts({ directory: release, metadataName: 'dev.yml', version, - artifactName: `Maka-${version}-win-x64.exe`, + artifactNames: [`Maka-${version}-win-x64.exe`], + }), + verifyDesktopUpdateArtifacts({ + directory: release, + metadataName: 'dev-linux.yml', + version, + artifactNames: [`Maka-${version}-linux-x86_64.AppImage`, `Maka-${version}-linux-amd64.deb`], + }), + verifyDesktopUpdateArtifacts({ + directory: release, + metadataName: 'dev-linux-arm64.yml', + version, + artifactNames: [`Maka-${version}-linux-arm64.AppImage`, `Maka-${version}-linux-arm64.deb`], }), ]); + // Both macOS architectures reach one feed, and the per-runner feeds do not + // reach the release at all. + const macFeed = parse(await readFile(join(release, 'dev-mac.yml'), 'utf8')); + assert.deepEqual( + macFeed.files.map((file) => file.url).sort(), + [`Maka-${version}-mac-arm64.zip`, `Maka-${version}-mac-x64.zip`].sort(), + ); + assert.deepEqual( (await readdir(release)).sort(), - [...payloadNames, 'dev-mac.yml', 'dev.yml'].sort(), + [...payloadNames, 'dev-mac.yml', 'dev.yml', 'dev-linux.yml', 'dev-linux-arm64.yml'].sort(), ); assert.deepEqual(await readdir(output), ['release']); }); diff --git a/scripts/desktop-nightly-workflow-policy.test.mjs b/scripts/desktop-nightly-workflow-policy.test.mjs index b604549cb9..5c561cf47d 100644 --- a/scripts/desktop-nightly-workflow-policy.test.mjs +++ b/scripts/desktop-nightly-workflow-policy.test.mjs @@ -22,6 +22,7 @@ import { spawnSync } from 'node:child_process'; import { readFile } from 'node:fs/promises'; import { test } from 'node:test'; import { parse } from 'yaml'; +import { desktopNightlyTargets } from './desktop-nightly.mjs'; async function readWorkflow(name) { return parse(await readFile(new URL(`../.github/workflows/${name}`, import.meta.url), 'utf8')); @@ -103,7 +104,7 @@ test('a failed Desktop Nightly is retried through a fresh npm Nightly', async () const download = workflow.jobs.publish.steps.find( (step) => step.uses?.startsWith('actions/download-artifact@') && step.with?.pattern, ); - assert.equal(upload.with.name, 'desktop-nightly-${{ matrix.platform }}'); + assert.equal(upload.with.name, 'desktop-nightly-${{ matrix.platform }}-${{ matrix.arch }}'); assert.equal(download.with.pattern, 'desktop-nightly-*'); }); @@ -115,9 +116,38 @@ test('Desktop Nightly packages the GitHub dev feeds and grants write only to its const stage = workflow.jobs.desktop.steps.find( (step) => step.name === 'Stage the exact Nightly artifacts', ); - assert.match(stage.run, /apps\/desktop\/release\/dev-mac\.yml/u); - assert.match(stage.run, /apps\/desktop\/release\/dev\.yml/u); - assert.doesNotMatch(stage.run, /latest-mac\.yml|latest\.yml/u); + // The runner never names its own uploads; the target descriptor does. + assert.match(stage.run, /desktop-nightly\.mjs stage-target/u); + assert.match(stage.run, /\$\{\{ matrix\.platform \}\}-\$\{\{ matrix\.arch \}\}/u); + assert.doesNotMatch(JSON.stringify(workflow), /latest-mac\.yml|latest\.yml/u); + // Nor does it name a distributable: the descriptor does, and the verifiers + // read it from there. + assert.doesNotMatch(JSON.stringify(workflow), /win-x64\.exe/u); +}); + +test('every packaged Desktop target ships from a runner of its own architecture', async () => { + const workflow = await readWorkflow('desktop-nightly.yml'); + const targets = workflow.jobs.desktop.strategy.matrix.include; + const names = targets.map((entry) => `${entry.platform}-${entry.arch}`); + assert.deepEqual(names, ['macos-arm64', 'macos-x64', 'windows-x64', 'linux-x64', 'linux-arm64']); + const nightlyTargets = desktopNightlyTargets('0.2.0-dev.42.20260829').map( + (target) => target.name, + ); + assert.deepEqual(names.toSorted(), nightlyTargets.toSorted()); + // The runner image is the workflow's to choose; what it may not do is choose + // one that disagrees with the row it builds. The native Runtime Host peer is + // never cross-built, so every row runs on its own platform and architecture. + for (const { platform, arch, runner } of targets) { + if (platform === 'macos') { + assert.match(runner, /^macos-/u); + assert.equal(runner.endsWith('-intel'), arch === 'x64', runner); + } else if (platform === 'windows') { + assert.match(runner, /^windows-/u); + } else { + assert.match(runner, /^ubuntu-/u); + assert.equal(runner.endsWith('-arm'), arch === 'arm64', runner); + } + } }); test('the publisher verifies exact GitHub identity and assets before publishing last', async () => { @@ -146,6 +176,21 @@ test('the publisher verifies exact GitHub identity and assets before publishing ); }); +test('the Nightly Linux verification runs under a virtual display', async () => { + // The last thing `verify:linux` does is launch the extracted AppImage's + // renderer. A headless runner has no display, so a step that dropped + // `xvfb-run` would fail every Nightly at its slowest point. + const workflow = await readWorkflow('desktop-nightly.yml'); + const steps = Object.values(workflow.jobs) + .flatMap((job) => job.steps ?? []) + .filter((step) => typeof step.run === 'string' && step.run.includes('npm run verify:linux')); + + assert.equal(steps.length, 1); + for (const step of steps) { + assert.match(step.run, /^xvfb-run\b/u, step.name); + } +}); + test('Desktop Nightly has no Apache Nightlies transport or compatibility state', async () => { const workflow = await readWorkflow('desktop-nightly.yml'); assert.equal(workflow.jobs.publish.environment, 'nightly'); diff --git a/scripts/desktop-nightly.mjs b/scripts/desktop-nightly.mjs index 2159552142..5e591b98ea 100644 --- a/scripts/desktop-nightly.mjs +++ b/scripts/desktop-nightly.mjs @@ -20,7 +20,11 @@ import { copyFile, mkdir, readFile, readdir, rm, stat } from 'node:fs/promises'; import { dirname, join } from 'node:path'; import { fileURLToPath, pathToFileURL } from 'node:url'; -import { verifyDesktopUpdateArtifacts } from './desktop-update-contract.mjs'; +import { desktopPublishedFeeds, desktopReleaseTargets } from './desktop-release-targets.mjs'; +import { + mergeDesktopUpdateFeeds, + verifyDesktopUpdateArtifacts, +} from './desktop-update-contract.mjs'; import { assertProductNightlyVersion } from './release-version.mjs'; const repoRoot = dirname(dirname(fileURLToPath(import.meta.url))); @@ -40,44 +44,109 @@ export function resolveRuntimeHostSetupPackage(productVersion, environment = pro return `maka-agent@${resolveDesktopBuildVersion(productVersion, environment)}`; } +/** + * The one place a packaging or verification step turns a target name into the + * files on disk. The version comes from the Desktop manifest and the nightly + * environment override, the names come from the descriptor, and the directory + * is the one electron-builder writes into — so no step spells an artifact name, + * and none of them can disagree about which channel it is building. + * + * It lives here rather than beside the descriptor because the descriptor cannot + * import `resolveDesktopBuildVersion` back out of this module without a cycle. + */ +export async function resolveDesktopReleaseTarget( + name, + { environment = process.env, root = repoRoot } = {}, +) { + const desktopRoot = join(root, 'apps', 'desktop'); + const manifest = JSON.parse(await readFile(join(desktopRoot, 'package.json'), 'utf8')); + const version = resolveDesktopBuildVersion(manifest.version, environment); + const nightly = version !== manifest.version; + const target = desktopReleaseTargets(version, { nightly }).find((entry) => entry.name === name); + if (!target) { + throw new Error(`Unknown Desktop release target ${name}.`); + } + const releaseDirectory = join(desktopRoot, 'release'); + return { + ...target, + version, + nightly, + releaseDirectory, + payloadPath(extension) { + const payload = target.payloads.find((payloadName) => payloadName.endsWith(extension)); + if (!payload) { + throw new Error(`Target ${name} ships no ${extension} payload.`); + } + return join(releaseDirectory, payload); + }, + checksumPaths() { + return target.checksums.map((checksumName) => join(releaseDirectory, checksumName)); + }, + }; +} + +export function desktopNightlyTargets(version) { + return desktopReleaseTargets(version, { nightly: true }); +} + +function desktopNightlyPublishedFeeds(version) { + return desktopPublishedFeeds(version, { nightly: true }); +} + +/** Everything the provenance attestation covers: the assets minus the bundle. */ +export function desktopNightlyAttestedAssetNames(version) { + return [ + ...desktopNightlyTargets(version).flatMap((target) => target.payloads), + ...desktopNightlyPublishedFeeds(version).map((feed) => feed.name), + ].sort(); +} + export function desktopNightlyReleaseAssetNames(version) { - const names = nightlyArtifactNames(version); return [ - names.macDmg, - names.macZip, - `${names.macZip}.blockmap`, - names.windowsExe, - `${names.windowsExe}.blockmap`, - names.windowsZip, + ...desktopNightlyAttestedAssetNames(version), `Maka-${version}-attestation.sigstore.json`, - 'dev-mac.yml', - 'dev.yml', ].sort(); } -function nightlyArtifactNames(version) { - return { - macZip: `Maka-${version}-mac-arm64.zip`, - macDmg: `Maka-${version}-mac-arm64.dmg`, - windowsExe: `Maka-${version}-win-x64.exe`, - windowsZip: `Maka-${version}-win-x64.zip`, - }; +/** + * Collects one packaging runner's uploads out of the build directory. The + * runner never names its own files: the target descriptor is the single place + * that knows what a target produces. + */ +export async function stageDesktopNightlyTarget({ + targetName, + releaseDirectory, + stageDirectory, + version, +}) { + const productManifest = JSON.parse(await readFile(join(repoRoot, 'package.json'), 'utf8')); + assertDesktopNightlyVersion(version, productManifest.version); + const target = desktopNightlyTargets(version).find((entry) => entry.name === targetName); + if (!target) { + throw new Error(`Unknown Desktop Nightly target: ${targetName}`); + } + await mkdir(stageDirectory, { recursive: true }); + const staged = [...target.payloads, target.feed]; + await Promise.all( + staged.map(async (name) => { + const sourcePath = join(releaseDirectory, name); + const info = await stat(sourcePath); + if (!info.isFile()) { + throw new Error(`Desktop Nightly payload is not a file: ${sourcePath}`); + } + await copyFile(sourcePath, join(stageDirectory, name)); + }), + ); + return staged; } export async function stageDesktopNightly({ inputDirectory, outputDirectory, version }) { const productManifest = JSON.parse(await readFile(join(repoRoot, 'package.json'), 'utf8')); assertDesktopNightlyVersion(version, productManifest.version); - const names = nightlyArtifactNames(version); - const payloads = [ - names.macDmg, - names.macZip, - `${names.macZip}.blockmap`, - names.windowsExe, - `${names.windowsExe}.blockmap`, - names.windowsZip, - ]; - const metadataNames = ['dev-mac.yml', 'dev.yml']; - const expected = [...payloads, ...metadataNames].sort(); + const targets = desktopNightlyTargets(version); + const feeds = desktopNightlyPublishedFeeds(version); + const payloads = targets.flatMap((target) => target.payloads); + const expected = [...payloads, ...targets.map((target) => target.feed)].sort(); const actual = (await readdir(inputDirectory)).sort(); if (JSON.stringify(actual) !== JSON.stringify(expected)) { throw new Error( @@ -85,32 +154,43 @@ export async function stageDesktopNightly({ inputDirectory, outputDirectory, ver ); } - await Promise.all([ - verifyDesktopUpdateArtifacts({ - directory: inputDirectory, - metadataName: 'dev-mac.yml', - version, - artifactName: names.macZip, - }), - verifyDesktopUpdateArtifacts({ - directory: inputDirectory, - metadataName: 'dev.yml', - version, - artifactName: names.windowsExe, - }), - ]); - await rm(outputDirectory, { recursive: true, force: true }); const releaseDirectory = join(outputDirectory, 'release'); await mkdir(releaseDirectory, { recursive: true }); await Promise.all( - [...payloads, ...metadataNames].map(async (name) => { + payloads.map(async (name) => { const source = join(inputDirectory, name); const info = await stat(source); if (!info.isFile()) throw new Error(`Desktop Nightly payload is not a file: ${source}`); await copyFile(source, join(releaseDirectory, name)); }), ); + await Promise.all( + feeds.map(async (feed) => { + const output = join(releaseDirectory, feed.name); + if (feed.mergedFrom) { + await mergeDesktopUpdateFeeds({ + sourcePaths: feed.mergedFrom.map((name) => join(inputDirectory, name)), + outputPath: output, + }); + return; + } + await copyFile(join(inputDirectory, feed.name), output); + }), + ); + + // Verify what is about to be published, not what arrived: the macOS feed only + // becomes whole in the staging directory. + await Promise.all( + feeds.map((feed) => + verifyDesktopUpdateArtifacts({ + directory: releaseDirectory, + metadataName: feed.name, + version, + artifactNames: feed.advertised, + }), + ), + ); } export async function addDesktopNightlyAttestation({ outputDirectory, version, bundlePath }) { @@ -136,13 +216,25 @@ async function main(args) { }); return; } + if (command === 'attested-assets' && rest.length === 1) { + const [version] = rest; + const productManifest = JSON.parse(await readFile(join(repoRoot, 'package.json'), 'utf8')); + assertDesktopNightlyVersion(version, productManifest.version); + process.stdout.write(`${desktopNightlyAttestedAssetNames(version).join('\n')}\n`); + return; + } + if (command === 'stage-target' && rest.length === 4) { + const [targetName, releaseDirectory, stageDirectory, version] = rest; + await stageDesktopNightlyTarget({ targetName, releaseDirectory, stageDirectory, version }); + return; + } if (command === 'add-attestation' && rest.length === 3) { const [outputDirectory, version, bundlePath] = rest; await addDesktopNightlyAttestation({ outputDirectory, version, bundlePath }); return; } throw new Error( - 'usage: desktop-nightly.mjs stage | add-attestation ', + 'usage: desktop-nightly.mjs stage | stage-target | add-attestation ', ); } diff --git a/scripts/desktop-nightly.test.mjs b/scripts/desktop-nightly.test.mjs index c65b595756..556fc2bf9d 100644 --- a/scripts/desktop-nightly.test.mjs +++ b/scripts/desktop-nightly.test.mjs @@ -25,8 +25,10 @@ import { basename, dirname, join } from 'node:path'; import { test } from 'node:test'; import { fileURLToPath } from 'node:url'; import { resolveDesktopBuilderConfig } from '../apps/desktop/electron-builder.config.mjs'; -import { packageMacosArm64 } from './package-macos-arm64.mjs'; +import { packageLinux } from './package-linux.mjs'; +import { packageMacos } from './package-macos.mjs'; import { packageWindowsX64 } from './package-windows-x64.mjs'; +import { desktopReleaseTargets } from './desktop-release-targets.mjs'; import { resolveDesktopBuildVersion, resolveRuntimeHostSetupPackage } from './desktop-nightly.mjs'; import { assertPackagedUpdateConfiguration } from './desktop-update-contract.mjs'; @@ -55,7 +57,11 @@ test('a nightly package embeds only the Apache GitHub dev update authority', () test('the macOS Nightly wrapper accepts dev update metadata', async () => { const version = '0.2.0-dev.42.20260829'; - await packageMacosArm64({ + const macosTarget = desktopReleaseTargets(version, { nightly: true }).find( + (entry) => entry.name === 'macos-arm64', + ); + const dmgPath = await packageMacos({ + targetArch: 'arm64', platform: 'darwin', arch: 'arm64', env: { @@ -68,15 +74,30 @@ test('the macOS Nightly wrapper accepts dev update metadata', async () => { }, run: async () => {}, remove: async () => {}, + move: async (source, destination) => { + // The feed leaves packaging under its architecture so the two macOS + // uploads cannot overwrite each other. + assert.equal(basename(source), 'dev-mac.yml'); + assert.equal(basename(destination), macosTarget.feed); + assert.equal(basename(destination), 'dev-mac-arm64.yml'); + }, assertFile: async (path) => { if (path.endsWith('.yml')) assert.equal(basename(path), 'dev-mac.yml'); }, }); + // Nothing here spells a distributable's name: the descriptor does. + assert.equal( + basename(dmgPath), + macosTarget.payloads.find((name) => name.endsWith('.dmg')), + ); }); test('the Windows Nightly wrapper accepts dev update metadata', async () => { const version = '0.2.0-dev.42.20260829'; - await packageWindowsX64({ + const windowsTarget = desktopReleaseTargets(version, { nightly: true }).find( + (entry) => entry.name === 'windows-x64', + ); + const { exePath, zipPath } = await packageWindowsX64({ platform: 'win32', arch: 'x64', env: { MAKA_DESKTOP_NIGHTLY_VERSION: version }, @@ -85,9 +106,55 @@ test('the Windows Nightly wrapper accepts dev update metadata', async () => { makeDirectory: async () => {}, copy: async () => {}, assertFile: async (path) => { - if (path.endsWith('.yml')) assert.equal(basename(path), 'dev.yml'); + if (path.endsWith('.yml')) { + assert.equal(basename(path), windowsTarget.feed); + assert.equal(basename(path), 'dev.yml'); + } + }, + }); + // Nothing here spells a distributable's name: the descriptor does. + assert.equal( + basename(exePath), + windowsTarget.payloads.find((name) => name.endsWith('.exe')), + ); + assert.equal( + basename(zipPath), + windowsTarget.payloads.find((name) => name.endsWith('.zip')), + ); +}); + +test('the Linux Nightly wrapper builds the AppImage before the deb and merges one feed', async () => { + const version = '0.2.0-dev.42.20260829'; + const events = []; + await packageLinux({ + platform: 'linux', + arch: 'x64', + env: { MAKA_DESKTOP_NIGHTLY_VERSION: version }, + run: async (_command, args) => { + const script = args.at(-1); + if (script.startsWith('package:')) events.push(script); + }, + remove: async () => {}, + move: async (source, destination) => { + events.push(`move ${basename(source)} ${basename(destination)}`); + }, + mergeFeeds: async ({ sourcePaths, outputPath }) => { + events.push( + `merge ${sourcePaths.map((path) => basename(path)).join(' ')} ${basename(outputPath)}`, + ); }, + assertFile: async () => {}, }); + // The deb run writes a `package-type` marker into the tree both targets share, + // and an AppImage carrying it updates itself by installing a deb. Building the + // AppImage first, in a run of its own, is the only thing keeping it out — and + // the second run rewrites the feed, so the two are merged back afterwards. + assert.deepEqual(events, [ + 'package:linux-appimage-x64', + 'move dev-linux.yml dev-linux.yml.appimage', + 'package:linux-deb-x64', + 'merge dev-linux.yml.appimage dev-linux.yml dev-linux.yml', + ]); }); test('a packaged Nightly accepts the pinned GitHub dev update channel', async () => { diff --git a/scripts/desktop-release-targets.mjs b/scripts/desktop-release-targets.mjs new file mode 100644 index 0000000000..3659de7a36 --- /dev/null +++ b/scripts/desktop-release-targets.mjs @@ -0,0 +1,114 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import { linuxUpdateMetadataName } from './desktop-update-contract.mjs'; + +const MACOS_ARCHITECTURES = Object.freeze(['arm64', 'x64']); +const LINUX_ARCHITECTURES = Object.freeze(['x64', 'arm64']); + +/** + * `${arch}` in a Linux `artifactName` is the packaging ecosystem's spelling, not + * the Node one: builder-util's getArtifactArchName rewrites x64 to `x86_64` for + * an AppImage and to `amd64` for a deb, and leaves arm64 alone. Recorded here + * because these names are what every other step looks for; the mapping itself is + * pinned to electron-builder's own function by desktop-release-targets.test.mjs. + */ +const LINUX_ARTIFACT_ARCHITECTURES = Object.freeze({ + x64: Object.freeze({ AppImage: 'x86_64', deb: 'amd64' }), + arm64: Object.freeze({ AppImage: 'arm64', deb: 'arm64' }), +}); + +/** + * Every Desktop packaging runner, for either publication channel. A target is + * one runner: `payloads` is what it uploads, `advertised` is the subset its + * update feed offers, and `checksums` is the subset a formal release publishes + * a `.sha256` beside. A checksum is issued only for a payload the platform's + * verify script has opened and accepted, and the macOS script opens the DMG + * alone — the ZIP is exercised end to end by the autoupdate verification, which + * issues none — so the set is recorded here rather than derived from `payloads`. + * + * `feed` is per runner. On macOS that is not the name clients read: both + * architectures build one feed between them, so the packaging step names each + * one after its architecture and publication merges them back. + */ +export function desktopReleaseTargets(version, { nightly }) { + const channel = nightly ? 'dev' : 'latest'; + const windowsExe = `Maka-${version}-win-x64.exe`; + const windowsZip = `Maka-${version}-win-x64.zip`; + return [ + ...MACOS_ARCHITECTURES.map((arch) => { + const dmg = `Maka-${version}-mac-${arch}.dmg`; + const zip = `Maka-${version}-mac-${arch}.zip`; + return { + name: `macos-${arch}`, + platform: 'macos', + arch, + payloads: [dmg, zip, `${zip}.blockmap`], + feed: `${channel}-mac-${arch}.yml`, + advertised: [zip], + checksums: [dmg], + }; + }), + { + name: 'windows-x64', + platform: 'windows', + arch: 'x64', + payloads: [windowsExe, `${windowsExe}.blockmap`, windowsZip], + feed: `${channel}.yml`, + advertised: [windowsExe], + checksums: [windowsExe, windowsZip], + }, + ...LINUX_ARCHITECTURES.map((arch) => { + const names = LINUX_ARTIFACT_ARCHITECTURES[arch]; + const appImage = `Maka-${version}-linux-${names.AppImage}.AppImage`; + const deb = `Maka-${version}-linux-${names.deb}.deb`; + // Neither Linux distributable has a sidecar blockmap: electron-builder + // appends the AppImage's block map to the AppImage itself, and fpm targets + // have none at all. + return { + name: `linux-${arch}`, + platform: 'linux', + arch, + payloads: [appImage, deb], + feed: linuxUpdateMetadataName(arch, nightly), + advertised: [appImage, deb], + checksums: [appImage, deb], + }; + }), + ]; +} + +/** + * The feeds a client actually reads. macOS is the one platform whose targets + * collapse into a single feed; the rest publish what their runner produced. + */ +export function desktopPublishedFeeds(version, { nightly }) { + const targets = desktopReleaseTargets(version, { nightly }); + const macos = targets.filter((target) => target.platform === 'macos'); + return [ + { + name: `${nightly ? 'dev' : 'latest'}-mac.yml`, + advertised: macos.flatMap((target) => target.advertised), + mergedFrom: macos.map((target) => target.feed), + }, + ...targets + .filter((target) => target.platform !== 'macos') + .map((target) => ({ name: target.feed, advertised: target.advertised, mergedFrom: null })), + ]; +} diff --git a/scripts/desktop-release-targets.test.mjs b/scripts/desktop-release-targets.test.mjs new file mode 100644 index 0000000000..2d3b19a5f0 --- /dev/null +++ b/scripts/desktop-release-targets.test.mjs @@ -0,0 +1,274 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import assert from 'node:assert/strict'; +import { readFile } from 'node:fs/promises'; +import { createRequire } from 'node:module'; +import test from 'node:test'; +import { fileURLToPath } from 'node:url'; +import { resolveDesktopBuilderConfig } from '../apps/desktop/electron-builder.config.mjs'; +import { desktopPublishedFeeds, desktopReleaseTargets } from './desktop-release-targets.mjs'; + +// The descriptor states what electron-builder produces. Asking electron-builder +// itself is the only way to know whether that statement is true, so this test +// drives its own resolution functions rather than restating their behaviour. +const require = createRequire(import.meta.url); +const { Packager } = require('app-builder-lib'); +const { + computeArchToTargetNamesMap, + createTargets, +} = require('app-builder-lib/out/targets/targetFactory'); +const { expandMacro } = require('app-builder-lib/out/util/macroExpander'); +const { Arch, getArtifactArchName } = require('builder-util/out/arch'); +const { normalizeOptions } = require('electron-builder/out/builder'); + +const VERSION = '9.8.7'; +const PLATFORM_FLAGS = new Set(['mac', 'win', 'linux']); +const ARCHITECTURE_FLAGS = new Set(['x64', 'arm64', 'ia32', 'armv7l', 'universal']); +/** The file extension each target emits, which is also the `${ext}` macro. */ +const TARGET_EXTENSIONS = Object.freeze({ + dmg: 'dmg', + zip: 'zip', + nsis: 'exe', + AppImage: 'AppImage', + deb: 'deb', +}); +const DESCRIPTOR_PLATFORMS = Object.freeze({ mac: 'macos', win: 'windows', linux: 'linux' }); + +/** + * Only the NSIS installer and the macOS ZIP get a `.blockmap` beside them: + * ArchiveTarget calls `createBlockmap` for a ZIP on macOS and `appendBlockmap` + * everywhere else, and NsisTarget calls `createBlockmap` for the installer. An + * AppImage carries its block map inside itself and fpm targets build none. + */ +function hasSidecarBlockmap(platformKey, targetName) { + if (targetName === 'nsis') return true; + return targetName === 'zip' && platformKey === 'mac'; +} + +/** yargs is not involved here, so the flags the packaging scripts use are parsed directly. */ +function parseElectronBuilderArguments(command) { + const tokens = command.split(/\s+/u); + assert.equal(tokens.shift(), 'electron-builder'); + const parsed = {}; + let openPlatform = null; + while (tokens.length > 0) { + const token = tokens.shift(); + if (!token.startsWith('--')) { + assert.ok(openPlatform, `unexpected positional ${token} in ${command}`); + parsed[openPlatform].push(token); + continue; + } + const name = token.slice(2); + openPlatform = null; + if (PLATFORM_FLAGS.has(name)) { + parsed[name] = []; + openPlatform = name; + } else if (ARCHITECTURE_FLAGS.has(name)) { + parsed[name] = true; + } else if (tokens[0] && !tokens[0].startsWith('--')) { + tokens.shift(); + } + } + return parsed; +} + +/** What one packaging script actually leaves in the release directory. */ +function producedArtifacts(command, configuration) { + const { targets } = normalizeOptions(parseElectronBuilderArguments(command)); + const produced = []; + for (const [platform, raw] of targets) { + const platformKey = platform.buildConfigurationKey; + const options = configuration[platformKey]; + const resolved = computeArchToTargetNamesMap( + raw, + { platformSpecificBuildOptions: options }, + platform, + ); + for (const [arch, targetNames] of resolved) { + for (const targetName of targetNames) { + const ext = TARGET_EXTENSIONS[targetName]; + assert.ok(ext, `unknown target ${targetName}`); + const pattern = options.artifactName ?? configuration.artifactName; + const name = expandMacro( + pattern, + getArtifactArchName(arch, ext), + { version: VERSION }, + { + ext, + }, + ); + produced.push({ platformKey, arch: Arch[arch], name }); + if (hasSidecarBlockmap(platformKey, targetName)) { + produced.push({ platformKey, arch: Arch[arch], name: `${name}.blockmap` }); + } + } + } + } + return produced; +} + +async function packagingScripts() { + const manifest = JSON.parse( + await readFile(new URL('../apps/desktop/package.json', import.meta.url), 'utf8'), + ); + return Object.entries(manifest.scripts).filter( + ([name, command]) => name.startsWith('package:') && command.startsWith('electron-builder'), + ); +} + +test('the packaging scripts produce exactly the payloads the target descriptor names', async () => { + const configuration = resolveDesktopBuilderConfig({}); + const scripts = await packagingScripts(); + assert.ok(scripts.length > 0); + + const produced = new Map(); + for (const [, command] of scripts) { + for (const artifact of producedArtifacts(command, configuration)) { + const target = `${DESCRIPTOR_PLATFORMS[artifact.platformKey]}-${artifact.arch}`; + if (!produced.has(target)) produced.set(target, []); + produced.get(target).push(artifact.name); + } + } + + const expected = new Map( + desktopReleaseTargets(VERSION, { nightly: false }).map((target) => [ + target.name, + [...target.payloads].sort(), + ]), + ); + assert.deepEqual( + [...produced.keys()].sort(), + [...expected.keys()].sort(), + 'a packaging script builds a target the descriptor does not describe, or the reverse', + ); + for (const [target, names] of produced) { + assert.deepEqual([...names].sort(), expected.get(target), `payloads of ${target}`); + } +}); + +test('no packaging script builds an architecture other than its own', async () => { + const configuration = resolveDesktopBuilderConfig({}); + for (const [name, command] of await packagingScripts()) { + const architectures = new Set( + producedArtifacts(command, configuration).map((artifact) => artifact.arch), + ); + // The native Runtime Host peer is built for the host, so a runner that also + // cross-built the other architecture would ship it the wrong peer — and, + // on macOS, one that was never notarized. + assert.equal(architectures.size, 1, `${name} builds ${[...architectures].join(' and ')}`); + assert.ok(name.endsWith(`-${[...architectures][0]}`), `${name} does not name its architecture`); + } +}); + +/** + * A Linux executable name reaches the desktop entry's `Exec=` and `Icon=`, so it + * has to be a name a shell and a desktop environment will accept. + */ +const LINUX_EXECUTABLE_NAME = /^[a-z0-9][a-z0-9+._-]*$/u; +const DESKTOP_PROJECT_DIRECTORY = fileURLToPath(new URL('../apps/desktop', import.meta.url)); + +/** + * electron-builder rewrites the configuration object it is handed — `normalizeFiles` + * turns `files` into records in place — and `resolveDesktopBuilderConfig` returns one + * shared module-level object outside the Nightly branch. Packaging calls it once, so + * this only bites a test that builds several packagers. + * + * Copied through JSON rather than `structuredClone`: everything electron-builder reads + * here has to survive serialization anyway, and the structured clone algorithm rejected + * this object on Node 24 while accepting it on 26. `beforePack` is the one function in + * the configuration; it is carried across by reference and never invoked by this test. + */ +function isolatedBuilderConfig() { + const { beforePack, ...rest } = resolveDesktopBuilderConfig({}); + return { ...JSON.parse(JSON.stringify(rest)), beforePack }; +} + +/** + * electron-builder settles the Linux executable name and the deb's metadata long + * before it packages anything, and both are drawn from places the packaging + * configuration never mentions: the executable name falls back to the npm package + * name — which is scoped here, so it is not a legal executable name — and fpm + * refuses to run without a project homepage the manifests do not declare. Neither + * failure needs a build to observe, and neither is visible on macOS or Windows. + * Asking electron-builder costs milliseconds on any host; learning it from a real + * Linux build costs a runner, and learning it on release day costs a release. + */ +test('electron-builder accepts the configuration for every Linux target', async () => { + const scripts = (await packagingScripts()).filter( + ([, command]) => parseElectronBuilderArguments(command).linux, + ); + assert.ok(scripts.length > 0, 'no Linux packaging script to check'); + + for (const [name, command] of scripts) { + const { targets } = normalizeOptions(parseElectronBuilderArguments(command)); + for (const [platform, raw] of targets) { + const packager = new Packager({ + targets, + projectDir: DESKTOP_PROJECT_DIRECTORY, + config: isolatedBuilderConfig(), + }); + await packager.validateConfig(); + const helper = await packager.createHelper(platform); + + assert.match( + helper.executableName, + LINUX_EXECUTABLE_NAME, + `${name} would install the executable as ${helper.executableName}`, + ); + // The AppImage target inherits an empty `checkOptions`, so the assertion + // above is the only one that covers it; fpm implements it and is where the + // packaging metadata — homepage, maintainer — is demanded. + for (const target of createTargets( + new Map(), + [...raw.values()].flat(), + packager.projectDir, + helper, + )) { + await target.checkOptions(); + } + } + } +}); + +test('every payload a runner uploads is advertised or accounted for by one feed', () => { + const targets = desktopReleaseTargets(VERSION, { nightly: false }); + const feeds = desktopPublishedFeeds(VERSION, { nightly: false }); + const advertised = new Set(feeds.flatMap((feed) => feed.advertised)); + for (const target of targets) { + for (const name of target.advertised) { + assert.ok(target.payloads.includes(name), `${target.name} advertises an unbuilt ${name}`); + assert.ok(advertised.has(name), `${name} is advertised by no published feed`); + } + for (const name of target.checksums) { + assert.ok(target.payloads.includes(name), `${target.name} checksums an unbuilt ${name}`); + } + } + // A merged feed exists only for macOS, and it must consume every feed the + // macOS runners write — publishing one alone would offer one architecture an + // update it cannot install. + const merged = feeds.filter((feed) => feed.mergedFrom); + assert.deepEqual( + merged.flatMap((feed) => feed.mergedFrom).toSorted(), + targets + .filter((target) => target.platform === 'macos') + .map((target) => target.feed) + .toSorted(), + ); +}); diff --git a/scripts/desktop-update-contract.mjs b/scripts/desktop-update-contract.mjs index d058dcc3a1..0bd56eb85c 100644 --- a/scripts/desktop-update-contract.mjs +++ b/scripts/desktop-update-contract.mjs @@ -19,7 +19,7 @@ import { createHash } from 'node:crypto'; import { createReadStream } from 'node:fs'; -import { readFile, stat } from 'node:fs/promises'; +import { readFile, stat, writeFile } from 'node:fs/promises'; import { createServer } from 'node:http'; import { join } from 'node:path'; import { parseProductReleaseVersion } from './release-version.mjs'; @@ -38,6 +38,25 @@ export const DESKTOP_NIGHTLY_UPDATE_PROVIDER = Object.freeze({ updaterCacheDirName: '@makadesktop-updater', }); +/** + * electron-builder suffixes the Linux feed with the architecture for everything + * except x64, so the two Linux architectures never share a feed the way the two + * macOS ones do. + */ +export function linuxUpdateMetadataName(arch, isNightly) { + const channel = isNightly ? 'dev' : 'latest'; + return arch === 'x64' ? `${channel}-linux.yml` : `${channel}-linux-${arch}.yml`; +} + +/** + * The payloads electron-builder writes a `.blockmap` beside. Only the + * archive and NSIS targets call `createBlockmap`, which writes that sidecar; the + * AppImage calls `appendBlockmap`, which puts the block map inside the AppImage, + * and fpm targets build none. Naming the two that have one keeps a payload with + * no sidecar from silently skipping the check. + */ +const SIDECAR_BLOCKMAP_EXTENSIONS = Object.freeze(['.exe', '.zip']); + /** A stable successor lets stable, alpha, and beta candidates use one feed contract. */ export function bumpedAutoupdateVersion(candidateVersion) { const { core, prerelease } = parseProductReleaseVersion(candidateVersion); @@ -82,16 +101,63 @@ export async function assertPackagedUpdateConfiguration( return configuration; } +/** + * macOS carries every architecture in one feed, and electron-updater picks its + * payload out of that one `files` list. Each architecture is packaged on a + * runner of its own, so each build writes a feed naming only its own zip; + * publishing either alone would offer one architecture an update it cannot + * install. The first document is the primary: its `path` and top-level digest + * survive the merge, and the rest contribute only their payloads. + */ +function mergeDesktopUpdateFeedDocuments(documents) { + const [primary, ...rest] = documents; + if (!primary) { + throw new Error('Desktop update feed merge requires at least one feed'); + } + for (const document of rest) { + if (document?.version !== primary.version) { + throw new Error( + `Desktop update feeds disagree on version: ${JSON.stringify(primary.version)} and ${JSON.stringify(document?.version)}`, + ); + } + } + const files = documents.flatMap((document) => document.files ?? []); + const urls = files.map((file) => file?.url); + if (new Set(urls).size !== urls.length) { + throw new Error(`Desktop update feeds advertise the same payload twice: ${urls.join(', ')}`); + } + return { ...primary, files }; +} + +export async function mergeDesktopUpdateFeeds({ sourcePaths, outputPath }) { + const documents = await Promise.all(sourcePaths.map((path) => readYaml(path))); + const merged = mergeDesktopUpdateFeedDocuments(documents); + const { stringify } = await import('yaml'); + await writeFile(outputPath, stringify(merged), 'utf8'); + return merged; +} + +async function sha512Base64(path) { + return new Promise((resolvePromise, reject) => { + const hash = createHash('sha512'); + const stream = createReadStream(path); + stream.once('error', reject); + stream.on('data', (chunk) => hash.update(chunk)); + stream.once('end', () => resolvePromise(hash.digest('base64'))); + }); +} + /** * Validates the update metadata against the bytes that will be published. - * The release is single-platform and single-architecture, so accepting extra - * payloads here would create an unverified update path. + * One feed carries every payload a client on that platform may be offered — + * both macOS architectures, or the AppImage and the deb — so the caller names + * all of them and anything else in the feed is an unverified update path. */ export async function verifyDesktopUpdateArtifacts({ directory, metadataName, version, - artifactName, + artifactNames, }) { const metadataPath = join(directory, metadataName); let metadata; @@ -105,36 +171,46 @@ export async function verifyDesktopUpdateArtifacts({ `${metadataName} advertises version ${JSON.stringify(metadata?.version)}, expected ${version}`, ); } - if (metadata.path !== artifactName || metadata.files?.length !== 1) { - throw new Error(`${metadataName} must advertise only ${artifactName}`); - } - const file = metadata.files[0]; - if (file?.url !== artifactName || file.sha512 !== metadata.sha512) { - throw new Error(`${metadataName} has inconsistent payload identity for ${artifactName}`); - } - const artifactPath = join(directory, artifactName); - const artifact = await stat(artifactPath); - if (!artifact.isFile()) throw new Error(`Desktop update payload is not a file: ${artifactPath}`); - const sha512 = await new Promise((resolvePromise, reject) => { - const hash = createHash('sha512'); - const stream = createReadStream(artifactPath); - stream.once('error', reject); - stream.on('data', (chunk) => hash.update(chunk)); - stream.once('end', () => resolvePromise(hash.digest('base64'))); - }); - if (metadata.sha512 !== sha512 || file.sha512 !== sha512) { - throw new Error(`${metadataName} sha512 does not match ${artifactName}`); + const expected = [...artifactNames].sort(); + const advertised = (metadata.files ?? []).map((file) => file?.url).sort(); + if (JSON.stringify(advertised) !== JSON.stringify(expected)) { + throw new Error( + `${metadataName} advertises ${JSON.stringify(advertised)}, expected ${JSON.stringify(expected)}`, + ); } - if (file.size !== artifact.size) { + if (!artifactNames.includes(metadata.path)) { throw new Error( - `${metadataName} records ${artifactName} size ${JSON.stringify(file.size)}, expected ${artifact.size}`, + `${metadataName} points at ${JSON.stringify(metadata.path)}, expected one of ${JSON.stringify(expected)}`, ); } - const blockmapPath = join(directory, `${artifactName}.blockmap`); - if (!(await stat(blockmapPath)).isFile()) { - throw new Error(`Desktop update blockmap is not a file: ${blockmapPath}`); + // The top-level digest belongs to the payload named by `path`; an updater + // that trusts it while downloading a different file would verify nothing. + const primary = metadata.files.find((file) => file.url === metadata.path); + if (primary.sha512 !== metadata.sha512) { + throw new Error(`${metadataName} has inconsistent payload identity for ${metadata.path}`); + } + + for (const file of metadata.files) { + const artifactPath = join(directory, file.url); + const artifact = await stat(artifactPath); + if (!artifact.isFile()) { + throw new Error(`Desktop update payload is not a file: ${artifactPath}`); + } + if (file.sha512 !== (await sha512Base64(artifactPath))) { + throw new Error(`${metadataName} sha512 does not match ${file.url}`); + } + if (file.size !== artifact.size) { + throw new Error( + `${metadataName} records ${file.url} size ${JSON.stringify(file.size)}, expected ${artifact.size}`, + ); + } + if (!SIDECAR_BLOCKMAP_EXTENSIONS.some((extension) => file.url.endsWith(extension))) continue; + const blockmapPath = join(directory, `${file.url}.blockmap`); + if (!(await stat(blockmapPath)).isFile()) { + throw new Error(`Desktop update blockmap is not a file: ${blockmapPath}`); + } } - return { artifactName, metadata, metadataName, version }; + return { artifactNames, metadata, metadataName, version }; } /** diff --git a/scripts/package-linux.mjs b/scripts/package-linux.mjs new file mode 100644 index 0000000000..86fcf774bc --- /dev/null +++ b/scripts/package-linux.mjs @@ -0,0 +1,137 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import { spawn } from 'node:child_process'; +import { access, rename, rm } from 'node:fs/promises'; +import { createRequire } from 'node:module'; +import { dirname, join } from 'node:path'; +import { fileURLToPath, pathToFileURL } from 'node:url'; +import { mergeDesktopUpdateFeeds } from './desktop-update-contract.mjs'; +import { resolveDesktopReleaseTarget } from './desktop-nightly.mjs'; + +const repoRoot = dirname(dirname(fileURLToPath(import.meta.url))); +const desktopRoot = join(repoRoot, 'apps', 'desktop'); +// electron is declared by apps/desktop, so resolve its install directory from +// there rather than assuming node_modules hoisted it to the repo root. +const require = createRequire(join(desktopRoot, 'package.json')); +const electronDistributionDirectory = join( + dirname(require.resolve('electron/package.json')), + 'dist', +); +const requiredElectronLicensePaths = [ + join(electronDistributionDirectory, 'LICENSE'), + join(electronDistributionDirectory, 'LICENSES.chromium.html'), +]; + +const linuxPackageArchitectures = Object.freeze(['x64', 'arm64']); + +/** electron-builder names the unpacked staging directory after the target. */ +function linuxUnpackedDirectoryName(arch) { + return arch === 'x64' ? 'linux-unpacked' : `linux-${arch}-unpacked`; +} + +function runCommand(command, args) { + return new Promise((resolve, reject) => { + const child = spawn(command, args, { + cwd: repoRoot, + env: process.env, + stdio: 'inherit', + }); + child.once('error', reject); + child.once('exit', (code, signal) => { + if (code === 0) { + resolve(); + return; + } + reject( + new Error( + `${command} ${args.join(' ')} failed with ${ + signal ? `signal ${signal}` : `exit code ${code}` + }`, + ), + ); + }); + }); +} + +export async function packageLinux({ + platform = process.platform, + arch = process.arch, + env = process.env, + run = runCommand, + remove = rm, + move = rename, + mergeFeeds = mergeDesktopUpdateFeeds, + assertFile = access, +} = {}) { + // The native Runtime Host peer is built for the host, so each architecture + // ships from a runner of its own rather than cross-building both from one. + if (platform !== 'linux' || !linuxPackageArchitectures.includes(arch)) { + throw new Error( + `Release packaging requires a Linux ${linuxPackageArchitectures.join(' or ')} host.`, + ); + } + + const target = await resolveDesktopReleaseTarget(`linux-${arch}`, { environment: env }); + const appImagePath = target.payloadPath('.AppImage'); + const debPath = target.payloadPath('.deb'); + const updateMetadataPath = join(target.releaseDirectory, target.feed); + // electron-builder rewrites the feed on every run rather than adding to it, so + // the AppImage's copy is moved aside and merged back once the deb has run. + const appImageMetadataPath = `${updateMetadataPath}.appimage`; + + for (const path of requiredElectronLicensePaths) { + await assertFile(path); + } + + await run('npm', ['run', 'clean']); + await run('npm', ['run', 'build']); + await run('npm', ['run', 'build:runtime-host-peer']); + await run('npm', ['run', 'check:runtime-host-peer-notices']); + await run('npm', ['run', 'check:release']); + await remove(target.releaseDirectory, { recursive: true, force: true }); + // Both distributables come out of one unpacked tree, and the deb target writes + // a `package-type` marker into it that makes electron-updater drive an install + // through DebUpdater. Building the AppImage in a run of its own is what keeps + // that marker out of the AppImage, which has to update itself in place. + await run('npm', ['--workspace', '@maka/desktop', 'run', `package:linux-appimage-${arch}`]); + await assertFile(appImagePath); + await assertFile(updateMetadataPath); + await move(updateMetadataPath, appImageMetadataPath); + await run('npm', ['--workspace', '@maka/desktop', 'run', `package:linux-deb-${arch}`]); + await assertFile(debPath); + await assertFile(updateMetadataPath); + await mergeFeeds({ + sourcePaths: [appImageMetadataPath, updateMetadataPath], + outputPath: updateMetadataPath, + }); + await remove(appImageMetadataPath, { force: true }); + await remove(join(target.releaseDirectory, linuxUnpackedDirectoryName(arch)), { + recursive: true, + force: true, + }); + + return { appImagePath, debPath, updateMetadataPath }; +} + +if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) { + const { appImagePath, debPath } = await packageLinux(); + console.log(`Created ${appImagePath}`); + console.log(`Created ${debPath}`); +} diff --git a/scripts/package-macos-autoupdate-next.mjs b/scripts/package-macos-autoupdate-next.mjs index 169734599c..84ad467abe 100644 --- a/scripts/package-macos-autoupdate-next.mjs +++ b/scripts/package-macos-autoupdate-next.mjs @@ -17,9 +17,11 @@ * under the License. */ -import { access, readFile, rm } from 'node:fs/promises'; +import { access, rm } from 'node:fs/promises'; import { dirname, join } from 'node:path'; import { fileURLToPath, pathToFileURL } from 'node:url'; +import { resolveDesktopReleaseTarget } from './desktop-nightly.mjs'; +import { desktopReleaseTargets } from './desktop-release-targets.mjs'; import { bumpedAutoupdateVersion, verifyDesktopUpdateArtifacts, @@ -39,13 +41,17 @@ export async function packageMacosAutoupdateNext({ if (platform !== 'darwin' || arch !== 'arm64') { throw new Error('The macOS auto-update build requires an Apple Silicon macOS host.'); } - const manifest = JSON.parse(await readFile(join(desktopRoot, 'package.json'), 'utf8')); - const nextVersion = bumpedAutoupdateVersion(manifest.version); + const candidate = await resolveDesktopReleaseTarget('macos-arm64', { environment: env }); + const nextVersion = bumpedAutoupdateVersion(candidate.version); const outputDirectory = join(desktopRoot, 'release-autoupdate-next'); - const zipName = `Maka-${nextVersion}-mac-arm64.zip`; + // The bumped build is the same target under a different version, so its ZIP is + // named by the same descriptor the candidate's is. + const zipName = desktopReleaseTargets(nextVersion, { nightly: false }) + .find((entry) => entry.name === 'macos-arm64') + .payloads.find((name) => name.endsWith('.zip')); await access(join(desktopRoot, 'dist')); - await access(join(desktopRoot, 'release', `Maka-${manifest.version}-mac-arm64.zip`)); + await access(candidate.payloadPath('.zip')); await rm(outputDirectory, { recursive: true, force: true }); const args = [ @@ -77,7 +83,7 @@ export async function packageMacosAutoupdateNext({ directory: outputDirectory, metadataName: 'latest-mac.yml', version: nextVersion, - artifactName: zipName, + artifactNames: [zipName], }); await rm(join(outputDirectory, 'mac-arm64'), { recursive: true, force: true }); return { diff --git a/scripts/package-macos-arm64.mjs b/scripts/package-macos.mjs similarity index 60% rename from scripts/package-macos-arm64.mjs rename to scripts/package-macos.mjs index 6b096d9be3..6d5d914ef8 100644 --- a/scripts/package-macos-arm64.mjs +++ b/scripts/package-macos.mjs @@ -18,15 +18,15 @@ */ import { spawn } from 'node:child_process'; -import { access, readFile, rm } from 'node:fs/promises'; +import { access, rename, rm } from 'node:fs/promises'; import { createRequire } from 'node:module'; import { dirname, join } from 'node:path'; import { fileURLToPath, pathToFileURL } from 'node:url'; -import { resolveDesktopBuildVersion } from './desktop-nightly.mjs'; +import { resolveDesktopReleaseTarget } from './desktop-nightly.mjs'; +import { desktopPublishedFeeds } from './desktop-release-targets.mjs'; const repoRoot = dirname(dirname(fileURLToPath(import.meta.url))); const desktopRoot = join(repoRoot, 'apps', 'desktop'); -const releaseDirectory = join(desktopRoot, 'release'); // electron is declared by apps/desktop, so resolve its install directory from // there rather than assuming node_modules hoisted it to the repo root. This // pre-flight guard exists to catch a missing electron dist before packaging; @@ -73,16 +73,28 @@ function runCommand(command, args) { }); } -export async function packageMacosArm64({ +const macosPackageArchitectures = Object.freeze(['arm64', 'x64']); + +export async function packageMacos({ + targetArch = process.arch, platform = process.platform, arch = process.arch, env = process.env, run = runCommand, remove = rm, + move = rename, assertFile = access, } = {}) { - if (platform !== 'darwin' || arch !== 'arm64') { - throw new Error('Release packaging requires an Apple Silicon macOS host.'); + if (!macosPackageArchitectures.includes(targetArch)) { + throw new Error( + `Release packaging supports ${macosPackageArchitectures.join(' and ')}, not ${targetArch}.`, + ); + } + // The native Runtime Host peer and the packaged smoke probes are built and + // run for the host, so each architecture ships from a runner of its own + // rather than cross-building both from one. + if (platform !== 'darwin' || arch !== targetArch) { + throw new Error(`Release packaging of ${targetArch} requires a ${targetArch} macOS host.`); } for (const name of requiredSigningEnvironment) { @@ -91,14 +103,20 @@ export async function packageMacosArm64({ } } - const manifest = JSON.parse(await readFile(join(desktopRoot, 'package.json'), 'utf8')); - const buildVersion = resolveDesktopBuildVersion(manifest.version, env); - const dmgPath = join(releaseDirectory, `Maka-${buildVersion}-mac-arm64.dmg`); - const zipPath = join(releaseDirectory, `Maka-${buildVersion}-mac-arm64.zip`); + const target = await resolveDesktopReleaseTarget(`macos-${targetArch}`, { environment: env }); + const dmgPath = target.payloadPath('.dmg'); + const zipPath = target.payloadPath('.zip'); + // Both architectures write the one feed clients read, and both uploads land + // in one directory before publication. Naming the feed after its architecture + // here is what keeps the two from overwriting each other; they are merged + // back into the single feed at publication time. const updateMetadataPath = join( - releaseDirectory, - buildVersion === manifest.version ? 'latest-mac.yml' : 'dev-mac.yml', + target.releaseDirectory, + desktopPublishedFeeds(target.version, { nightly: target.nightly }).find((feed) => + feed.mergedFrom?.includes(target.feed), + ).name, ); + const architectureMetadataPath = join(target.releaseDirectory, target.feed); for (const path of requiredElectronLicensePaths) { await assertFile(path); @@ -109,17 +127,23 @@ export async function packageMacosArm64({ await run('npm', ['run', 'build:runtime-host-peer']); await run('npm', ['run', 'check:runtime-host-peer-notices']); await run('npm', ['run', 'check:release']); - await remove(releaseDirectory, { recursive: true, force: true }); - await run('npm', ['--workspace', '@maka/desktop', 'run', 'package:macos-arm64']); + await remove(target.releaseDirectory, { recursive: true, force: true }); + await run('npm', ['--workspace', '@maka/desktop', 'run', `package:macos-${targetArch}`]); await assertFile(dmgPath); await assertFile(zipPath); await assertFile(updateMetadataPath); - await remove(join(releaseDirectory, 'mac-arm64'), { recursive: true, force: true }); + await move(updateMetadataPath, architectureMetadataPath); + // electron-builder names the unpacked staging directory after the target: + // `mac` for x64, `mac-` for everything else. + await remove(join(target.releaseDirectory, targetArch === 'x64' ? 'mac' : `mac-${targetArch}`), { + recursive: true, + force: true, + }); return dmgPath; } if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) { - const dmgPath = await packageMacosArm64(); + const dmgPath = await packageMacos({ targetArch: process.argv[2] || process.arch }); console.log(`Created ${dmgPath}`); } diff --git a/scripts/package-windows-autoupdate-next.mjs b/scripts/package-windows-autoupdate-next.mjs index c48b783ab6..a33fd230f7 100644 --- a/scripts/package-windows-autoupdate-next.mjs +++ b/scripts/package-windows-autoupdate-next.mjs @@ -21,6 +21,7 @@ import { access, readFile, rm } from 'node:fs/promises'; import { dirname, join } from 'node:path'; import { fileURLToPath, pathToFileURL } from 'node:url'; import { runCommand } from './package-windows-x64.mjs'; +import { desktopReleaseTargets } from './desktop-release-targets.mjs'; import { bumpedAutoupdateVersion, verifyDesktopUpdateArtifacts, @@ -59,7 +60,11 @@ export async function packageWindowsAutoupdateNext({ const manifest = JSON.parse(await readFile(join(desktopRoot, 'package.json'), 'utf8')); const nextVersion = bumpedAutoupdateVersion(manifest.version); const outputDirectory = join(desktopRoot, 'release-autoupdate-next'); - const exeName = `Maka-${nextVersion}-win-x64.exe`; + // The bumped build is the same target under a different version, so its + // installer is named by the same descriptor the real one is. + const exeName = desktopReleaseTargets(nextVersion, { nightly: false }) + .find((entry) => entry.name === 'windows-x64') + .payloads.find((name) => name.endsWith('.exe')); const exePath = join(outputDirectory, exeName); const latestYmlPath = join(outputDirectory, 'latest.yml'); const blockmapPath = join(outputDirectory, `${exeName}.blockmap`); @@ -94,7 +99,7 @@ export async function packageWindowsAutoupdateNext({ directory: outputDirectory, metadataName: 'latest.yml', version: nextVersion, - artifactName: exeName, + artifactNames: [exeName], }); return { version: nextVersion, exePath, latestYmlPath, blockmapPath, directory: outputDirectory }; diff --git a/scripts/package-windows-x64.mjs b/scripts/package-windows-x64.mjs index c7d02cd526..c3194af0f2 100644 --- a/scripts/package-windows-x64.mjs +++ b/scripts/package-windows-x64.mjs @@ -18,16 +18,15 @@ */ import { spawn } from 'node:child_process'; -import { access, copyFile, mkdir, readFile, rm } from 'node:fs/promises'; +import { access, copyFile, mkdir, rm } from 'node:fs/promises'; import { createRequire } from 'node:module'; import { dirname, join } from 'node:path'; import { fileURLToPath, pathToFileURL } from 'node:url'; -import { resolveDesktopBuildVersion } from './desktop-nightly.mjs'; +import { resolveDesktopReleaseTarget } from './desktop-nightly.mjs'; import { npmSpawnOptions } from './npm-spawn.mjs'; const repoRoot = dirname(dirname(fileURLToPath(import.meta.url))); const desktopRoot = join(repoRoot, 'apps', 'desktop'); -const releaseDirectory = join(desktopRoot, 'release'); // electron is declared by apps/desktop, so resolve its install directory from // there rather than assuming node_modules hoisted it to the repo root. This // pre-flight guard exists to catch a missing electron dist before packaging; @@ -105,15 +104,11 @@ export async function packageWindowsX64({ throw new Error('Release packaging requires a Windows x64 host.'); } - const manifest = JSON.parse(await readFile(join(desktopRoot, 'package.json'), 'utf8')); - const buildVersion = resolveDesktopBuildVersion(manifest.version, env); - const exePath = join(releaseDirectory, `Maka-${buildVersion}-win-x64.exe`); - const zipPath = join(releaseDirectory, `Maka-${buildVersion}-win-x64.zip`); - const updateMetadataPath = join( - releaseDirectory, - buildVersion === manifest.version ? 'latest.yml' : 'dev.yml', - ); - const unpackedDirectory = join(releaseDirectory, 'win-unpacked'); + const target = await resolveDesktopReleaseTarget('windows-x64', { environment: env }); + const exePath = target.payloadPath('.exe'); + const zipPath = target.payloadPath('.zip'); + const updateMetadataPath = join(target.releaseDirectory, target.feed); + const unpackedDirectory = join(target.releaseDirectory, 'win-unpacked'); for (const path of requiredElectronLicensePaths) { await assertFile(path); @@ -128,7 +123,7 @@ export async function packageWindowsX64({ await makeDirectory(sandboxResourceDirectory, { recursive: true }); await copy(sandboxBinaryPath, sandboxResourcePath); await run('npm', ['run', 'check:release']); - await remove(releaseDirectory, { recursive: true, force: true }); + await remove(target.releaseDirectory, { recursive: true, force: true }); await run('npm', ['--workspace', '@maka/desktop', 'run', 'package:windows-x64']); await assertFile(exePath); await assertFile(zipPath); diff --git a/scripts/product-release-artifacts.mjs b/scripts/product-release-artifacts.mjs index ae2596dfe5..d2de1a6b9f 100644 --- a/scripts/product-release-artifacts.mjs +++ b/scripts/product-release-artifacts.mjs @@ -19,10 +19,13 @@ import { createHash } from 'node:crypto'; import { createReadStream } from 'node:fs'; -import { copyFile, mkdir, readFile, readdir, stat, writeFile } from 'node:fs/promises'; +import { copyFile, mkdir, readFile, readdir, rm, stat, writeFile } from 'node:fs/promises'; import { join } from 'node:path'; import { pathToFileURL } from 'node:url'; -import { verifyDesktopUpdateArtifacts } from './desktop-update-contract.mjs'; +import { + mergeDesktopUpdateFeeds, + verifyDesktopUpdateArtifacts, +} from './desktop-update-contract.mjs'; import { parseAsfSourceReferenceTag, readProductReleaseIdentity, @@ -96,6 +99,26 @@ export async function verifyProductReleaseArtifactDirectory(directory, expectedN return assertExactArtifactSet(await regularFileNames(directory), expectedNames); } +/** + * A macOS client reads one feed covering both architectures, but each + * architecture is packaged on a runner of its own and can only write a feed + * naming its own payload. Publication is where the two become the feed clients + * read; the per-architecture copies are consumed here and never published. + */ +export async function mergeProductReleaseUpdateFeeds(directory, identity) { + const merged = []; + for (const feed of identity.updateFeeds) { + if (!feed.mergedFrom) continue; + await mergeDesktopUpdateFeeds({ + sourcePaths: feed.mergedFrom.map((name) => join(directory, name)), + outputPath: join(directory, feed.name), + }); + await Promise.all(feed.mergedFrom.map((name) => rm(join(directory, name)))); + merged.push(feed.name); + } + return merged; +} + function digestFile(path, algorithm = 'sha256') { return new Promise((resolvePromise, reject) => { const hash = createHash(algorithm); @@ -120,8 +143,8 @@ export async function productReleaseArtifactRecords(directory, names) { } export async function verifyProductReleaseArtifactIntegrity(directory, identity) { - await verifyProductReleaseArtifactDirectory(directory, allArtifactNames(identity)); - const checksumNames = allArtifactNames(identity).filter((name) => name.endsWith('.sha256')); + await verifyProductReleaseArtifactDirectory(directory, identity.releaseAssets); + const checksumNames = identity.releaseAssets.filter((name) => name.endsWith('.sha256')); for (const checksumName of checksumNames) { const artifactName = checksumName.slice(0, -'.sha256'.length); const source = await readFile(join(directory, checksumName), 'utf8'); @@ -134,25 +157,17 @@ export async function verifyProductReleaseArtifactIntegrity(directory, identity) throw new Error(`Product release checksum does not match: ${artifactName}`); } } - await Promise.all([ - verifyDesktopUpdateArtifacts({ - directory, - metadataName: 'latest-mac.yml', - version: identity.version, - artifactName: `Maka-${identity.version}-mac-arm64.zip`, - }), - verifyDesktopUpdateArtifacts({ - directory, - metadataName: 'latest.yml', - version: identity.version, - artifactName: identity.exe, - }), - ]); - return allArtifactNames(identity); -} - -function allArtifactNames(identity) { - return Object.values(identity.artifacts).flat(); + await Promise.all( + identity.updateFeeds.map((feed) => + verifyDesktopUpdateArtifacts({ + directory, + metadataName: feed.name, + version: identity.version, + artifactNames: feed.advertised, + }), + ), + ); + return identity.releaseAssets; } function exactKeys(value, expected, label) { @@ -259,7 +274,7 @@ export async function createProductReleasePublicationRecord({ sourceCommit: identity.sourceCommit, tag: identity.tag, version: identity.version, - assets: await productReleaseArtifactRecords(artifactDirectory, allArtifactNames(identity)), + assets: await productReleaseArtifactRecords(artifactDirectory, identity.releaseAssets), }); } @@ -301,6 +316,14 @@ if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) } await stageProductReleaseArtifactGroup({ sourceDirectory, targetDirectory, expectedNames }); console.log(`Staged exact ${group} product artifacts in ${targetDirectory}`); + } else if (command === 'merge-feeds') { + const identity = await readProductReleaseIdentity(); + const [directory] = args; + if (!directory) { + throw new Error('usage: product-release-artifacts.mjs merge-feeds '); + } + const merged = await mergeProductReleaseUpdateFeeds(directory, identity); + console.log(`Merged the per-architecture update feeds into ${merged.join(', ')}`); } else if (command === 'verify') { const identity = await readProductReleaseIdentity(); const [directory] = args; @@ -356,10 +379,10 @@ if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) console.log(`Verified immutable product release evidence for ${tag}`); } else if (command === 'list' && args.length === 0) { const identity = await readProductReleaseIdentity(); - console.log(JSON.stringify(identity.artifacts, null, 2)); + console.log(JSON.stringify(identity.releaseAssets, null, 2)); } else { throw new Error( - 'usage: product-release-artifacts.mjs ...', + 'usage: product-release-artifacts.mjs ...', ); } } diff --git a/scripts/product-release-authority.test.mjs b/scripts/product-release-authority.test.mjs index 55a48d1d02..5374a5fa1e 100644 --- a/scripts/product-release-authority.test.mjs +++ b/scripts/product-release-authority.test.mjs @@ -76,6 +76,11 @@ test('publication verifies live asset digests before one Stable/Latest mutation' sourceCommit, exe, artifacts: { test: names }, + releaseAssets: names, + updateFeeds: [ + { name: 'latest-mac.yml', advertised: [macZip] }, + { name: 'latest.yml', advertised: [exe] }, + ], }, repository: 'apache/maka', runId: '123', diff --git a/scripts/product-release-identity.mjs b/scripts/product-release-identity.mjs index 9464d3d806..9af90c72c8 100644 --- a/scripts/product-release-identity.mjs +++ b/scripts/product-release-identity.mjs @@ -22,6 +22,7 @@ import { appendFile, readFile } from 'node:fs/promises'; import { dirname, join } from 'node:path'; import { fileURLToPath, pathToFileURL } from 'node:url'; import { promisify } from 'node:util'; +import { desktopPublishedFeeds, desktopReleaseTargets } from './desktop-release-targets.mjs'; import { parseProductReleaseVersion } from './release-version.mjs'; const repoRoot = dirname(dirname(fileURLToPath(import.meta.url))); @@ -119,23 +120,35 @@ export function resolveProductReleaseIdentity({ } const toolchain = releaseToolchainFromManifest(rootManifest); - const dmg = `Maka-${version}-mac-arm64.dmg`; - const macZip = `Maka-${version}-mac-arm64.zip`; - const exe = `Maka-${version}-win-x64.exe`; - const windowsZip = `Maka-${version}-win-x64.zip`; + const targets = desktopReleaseTargets(version, { nightly: false }); const cliArchive = `Maka-${version}-cli-mac-arm64.zip`; + const cliGroup = [cliArchive, `${cliArchive}.sha256`]; + const uploaded = (target) => [ + ...target.payloads, + ...target.checksums.map((name) => `${name}.sha256`), + ]; + // What each runner uploads, keyed by its staging group. The macOS runners + // upload their feeds under per-architecture names; the release publishes the + // one merged feed instead, so the staged set and the published set differ. const artifacts = { - 'desktop-macos': [dmg, `${dmg}.sha256`, macZip, `${macZip}.blockmap`, 'latest-mac.yml'], - 'desktop-windows': [ - exe, - `${exe}.blockmap`, - `${exe}.sha256`, - windowsZip, - `${windowsZip}.sha256`, - 'latest.yml', - ], - 'cli-macos-arm64': [cliArchive, `${cliArchive}.sha256`], + ...Object.fromEntries( + targets.map((target) => [ + `desktop-${target.name}`, + [...uploaded(target), target.feed].sort(), + ]), + ), + 'cli-macos-arm64': cliGroup, }; + // Each feed carries its own `mergedFrom`, because publication is the only + // step that can turn the per-architecture copies the runners upload into the + // one feed `releaseAssets` names, and it needs to be told which copies those + // are. + const updateFeeds = desktopPublishedFeeds(version, { nightly: false }); + const releaseAssets = [ + ...targets.flatMap(uploaded), + ...updateFeeds.map((feed) => feed.name), + ...cliGroup, + ].sort(); return { ...toolchain, @@ -143,10 +156,10 @@ export function resolveProductReleaseIdentity({ tag: `v${version}`, sourceCommit: sha, sourceReferenceTag, - dmg, - exe, cliArchive, artifacts, + releaseAssets, + updateFeeds, }; } @@ -188,8 +201,6 @@ function githubOutputEntries(identity) { tag: identity.tag, source_commit: identity.sourceCommit, source_reference_tag: identity.sourceReferenceTag, - dmg: identity.dmg, - exe: identity.exe, cli_archive: identity.cliArchive, node_version: identity.nodeVersion, node_archive: identity.nodeArchive, diff --git a/scripts/product-release.test.mjs b/scripts/product-release.test.mjs index 9d20b28c7e..73341b6432 100644 --- a/scripts/product-release.test.mjs +++ b/scripts/product-release.test.mjs @@ -19,6 +19,7 @@ import assert from 'node:assert/strict'; import { execFile } from 'node:child_process'; +import { createHash } from 'node:crypto'; import { access, chmod, mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { isAbsolute, join } from 'node:path'; @@ -26,8 +27,17 @@ import test from 'node:test'; import { promisify } from 'node:util'; import { parse as parseYaml } from 'yaml'; import { resolveDesktopBuilderConfig } from '../apps/desktop/electron-builder.config.mjs'; +import { writeDesktopReleaseInput } from './desktop-nightly-fixture.mjs'; +import { desktopReleaseTargets } from './desktop-release-targets.mjs'; +import { verifyDesktopUpdateArtifacts } from './desktop-update-contract.mjs'; +import { + mergeProductReleaseUpdateFeeds, + verifyProductReleaseArtifactDirectory, + verifyProductReleaseArtifactIntegrity, +} from './product-release-artifacts.mjs'; import { parseAsfSourceReferenceTag, + readProductReleaseIdentity, resolveProductManifestIdentity, resolveProductReleaseIdentity, } from './product-release-identity.mjs'; @@ -92,19 +102,27 @@ test('one root version defines every product artifact from one source commit', ( 'https://nodejs.org/download/release/v24.18.1/node-v24.18.1-darwin-arm64.tar.xz', ); assert.equal(identity.npmVersion, '11.19.0'); - assert.equal(identity.dmg, 'Maka-1.2.3-mac-arm64.dmg'); - assert.equal(identity.exe, 'Maka-1.2.3-win-x64.exe'); assert.equal(identity.cliArchive, 'Maka-1.2.3-cli-mac-arm64.zip'); assert.equal(Object.hasOwn(identity, 'sourceArchive'), false); + // Desktop artifact names come from the target descriptor alone; the identity + // restates none of them. + assert.equal(Object.hasOwn(identity, 'exe'), false); assert.deepEqual(identity.artifacts, { - 'desktop-macos': [ + 'desktop-macos-arm64': [ 'Maka-1.2.3-mac-arm64.dmg', 'Maka-1.2.3-mac-arm64.dmg.sha256', 'Maka-1.2.3-mac-arm64.zip', 'Maka-1.2.3-mac-arm64.zip.blockmap', - 'latest-mac.yml', + 'latest-mac-arm64.yml', + ], + 'desktop-macos-x64': [ + 'Maka-1.2.3-mac-x64.dmg', + 'Maka-1.2.3-mac-x64.dmg.sha256', + 'Maka-1.2.3-mac-x64.zip', + 'Maka-1.2.3-mac-x64.zip.blockmap', + 'latest-mac-x64.yml', ], - 'desktop-windows': [ + 'desktop-windows-x64': [ 'Maka-1.2.3-win-x64.exe', 'Maka-1.2.3-win-x64.exe.blockmap', 'Maka-1.2.3-win-x64.exe.sha256', @@ -112,8 +130,176 @@ test('one root version defines every product artifact from one source commit', ( 'Maka-1.2.3-win-x64.zip.sha256', 'latest.yml', ], + // Linux artifact names carry the packaging ecosystem's architecture, not + // Node's: x64 is `x86_64` for an AppImage and `amd64` for a deb. + 'desktop-linux-x64': [ + 'Maka-1.2.3-linux-amd64.deb', + 'Maka-1.2.3-linux-amd64.deb.sha256', + 'Maka-1.2.3-linux-x86_64.AppImage', + 'Maka-1.2.3-linux-x86_64.AppImage.sha256', + 'latest-linux.yml', + ], + 'desktop-linux-arm64': [ + 'Maka-1.2.3-linux-arm64.AppImage', + 'Maka-1.2.3-linux-arm64.AppImage.sha256', + 'Maka-1.2.3-linux-arm64.deb', + 'Maka-1.2.3-linux-arm64.deb.sha256', + 'latest-linux-arm64.yml', + ], 'cli-macos-arm64': ['Maka-1.2.3-cli-mac-arm64.zip', 'Maka-1.2.3-cli-mac-arm64.zip.sha256'], }); + // The two macOS runners upload per-architecture feeds; the release carries + // the one merged feed clients read. + assert.ok(identity.releaseAssets.includes('latest-mac.yml')); + assert.ok(!identity.releaseAssets.includes('latest-mac-arm64.yml')); + assert.ok(!identity.releaseAssets.includes('latest-mac-x64.yml')); + // Nothing may reach publication that no runner uploads, and nothing a runner + // uploads may go unaccounted for. The merged feeds are the one difference, + // and each names exactly the per-architecture copies it consumes. + const staged = new Set(Object.values(identity.artifacts).flat()); + const published = new Set(identity.releaseAssets); + const consumed = new Set(identity.updateFeeds.flatMap((feed) => feed.mergedFrom ?? [])); + const produced = new Set( + identity.updateFeeds.filter((feed) => feed.mergedFrom).map((feed) => feed.name), + ); + assert.deepEqual( + [...published].filter((name) => !staged.has(name)).toSorted(), + [...produced].toSorted(), + ); + assert.deepEqual( + [...staged].filter((name) => !published.has(name)).toSorted(), + [...consumed].toSorted(), + ); +}); + +test('the manifest operators compare the Draft against is the published set', async () => { + // The release checklist reads this list against the Draft's assets, so it has + // to name what publication carries, not the per-architecture groups the + // runners stage and the publish job merges away. + const [{ stdout }, identity] = await Promise.all([ + execFileAsync( + process.execPath, + [join(repoRoot, 'scripts/product-release-artifacts.mjs'), 'list'], + { cwd: repoRoot }, + ), + readProductReleaseIdentity(), + ]); + + assert.deepEqual(JSON.parse(stdout), identity.releaseAssets); +}); + +test('publication merges the per-architecture macOS feeds into the one clients read', async (t) => { + const directory = await mkdtemp(join(tmpdir(), 'maka-merge-feeds-')); + t.after(() => rm(directory, { recursive: true, force: true })); + const version = '1.2.3'; + const feeds = []; + for (const arch of ['arm64', 'x64']) { + const zip = `Maka-${version}-mac-${arch}.zip`; + const bytes = Buffer.from(`${zip} bytes`); + const sha512 = createHash('sha512').update(bytes).digest('base64'); + await writeFile(join(directory, zip), bytes); + await writeFile(join(directory, `${zip}.blockmap`), 'blockmap'); + const name = `latest-mac-${arch}.yml`; + feeds.push(name); + await writeFile( + join(directory, name), + [ + `version: ${version}`, + 'files:', + ` - url: ${zip}`, + ` sha512: ${sha512}`, + ` size: ${bytes.length}`, + `path: ${zip}`, + `sha512: ${sha512}`, + '', + ].join('\n'), + ); + } + + const merged = await mergeProductReleaseUpdateFeeds(directory, { + updateFeeds: [ + { name: 'latest-mac.yml', advertised: [], mergedFrom: feeds }, + { name: 'latest.yml', advertised: [], mergedFrom: null }, + ], + }); + + assert.deepEqual(merged, ['latest-mac.yml']); + // The per-architecture copies are consumed, not published: leaving them would + // publish two feeds each offering one architecture an update it cannot use. + for (const name of feeds) { + await assert.rejects(access(join(directory, name))); + } + // The merged feed has to hold against the bytes that will be published, not + // merely parse: this is the only check between the two runners and a client. + await verifyDesktopUpdateArtifacts({ + directory, + metadataName: 'latest-mac.yml', + version, + artifactNames: [`Maka-${version}-mac-arm64.zip`, `Maka-${version}-mac-x64.zip`], + }); +}); + +test('the release Linux verification runs under a virtual display', async () => { + // The last thing `verify:linux` does is launch the extracted AppImage's + // renderer. A headless runner has no display, so a step that dropped + // `xvfb-run` would fail the whole release at its slowest point. + const workflow = parseYaml( + await readFile(new URL('../.github/workflows/release.yml', import.meta.url), 'utf8'), + ); + const steps = Object.values(workflow.jobs) + .flatMap((job) => job.steps ?? []) + .filter((step) => typeof step.run === 'string' && step.run.includes('npm run verify:linux')); + + assert.equal(steps.length, 1); + for (const step of steps) { + assert.match(step.run, /^xvfb-run\b/u, step.name); + } +}); + +test('the publish chain turns every staged group into the exact release assets', async (t) => { + // `release.yml` has never run, so until this existed nothing had executed the + // publish job's merge and verify over the whole staged set — only the macOS + // feed merge above, against a hand-written identity. This builds what all six + // runners upload and runs the two steps in the order the job runs them. + const identity = await readProductReleaseIdentity(); + const directory = await mkdtemp(join(tmpdir(), 'maka-release-assets-')); + t.after(() => rm(directory, { recursive: true, force: true })); + + // The desktop payloads, their blockmaps and each runner's own feed; the + // checksum sidecars and the CLI archive are the rest of the staged set. + await writeDesktopReleaseInput(directory, identity.version); + const packaged = new Set( + desktopReleaseTargets(identity.version, { nightly: false }).flatMap((target) => [ + ...target.payloads, + target.feed, + ]), + ); + const staged = [...new Set(Object.values(identity.artifacts).flat())].sort(); + for (const name of staged) { + if (packaged.has(name) || name.endsWith('.sha256')) continue; + await writeFile(join(directory, name), `${name} bytes`); + } + for (const name of staged.filter((entry) => entry.endsWith('.sha256'))) { + const artifact = name.slice(0, -'.sha256'.length); + const digest = createHash('sha256') + .update(await readFile(join(directory, artifact))) + .digest('hex'); + await writeFile(join(directory, name), `${digest} ${artifact}\n`); + } + + assert.deepEqual(await verifyProductReleaseArtifactDirectory(directory, staged), staged); + const merged = await mergeProductReleaseUpdateFeeds(directory, identity); + assert.deepEqual(merged, ['latest-mac.yml']); + assert.deepEqual( + await verifyProductReleaseArtifactIntegrity(directory, identity), + identity.releaseAssets, + ); + // What the publish job hands to the Draft is this directory, so nothing may + // remain in it that `releaseAssets` does not name. + assert.deepEqual( + await verifyProductReleaseArtifactDirectory(directory, identity.releaseAssets), + [...identity.releaseAssets].sort(), + ); }); test('the standalone launcher identifies its installed Eval bundle root', async (t) => { @@ -289,14 +475,20 @@ test('platform package verifiers keep Git checks out of every artifact', async ( assert.doesNotMatch(windowsSource, /requirePath\(join\(resources, ['"]git['"]/u); assert.match( windowsSource, - /if \(requiresCurrentContract\) \{\s*await assertPackagedUpdateConfiguration\(resources, \{\s*channel: environment\.MAKA_DESKTOP_NIGHTLY_VERSION \? ['"]nightly['"] : ['"]release['"],\s*\}\);\s*await assertPackagedDependencyClosure\(resources\);\s*\}/u, + /if \(requiresCurrentContract\) \{\s*await assertPackagedUpdateConfiguration\(resources, \{ channel \}\);\s*await assertPackagedDependencyClosure\(resources\);\s*\}/u, ); - const macosSource = await readFile( - join(repoRoot, 'scripts', 'verify-macos-arm64-dmg.mjs'), - 'utf8', - ); + const macosSource = await readFile(join(repoRoot, 'scripts', 'verify-macos-dmg.mjs'), 'utf8'); assert.doesNotMatch(macosSource, /requirePath\(join\(resources, ['"]git['"]/u); + + // The channel is the release descriptor's, resolved once beside the artifact + // names. Reading the nightly environment variable a second time here would + // let an unvalidated value disagree with the payloads that were just named. + const linuxSource = await readFile(join(repoRoot, 'scripts', 'verify-linux.mjs'), 'utf8'); + for (const source of [windowsSource, macosSource, linuxSource]) { + assert.doesNotMatch(source, /MAKA_DESKTOP_NIGHTLY_VERSION/u); + assert.match(source, /target\.nightly \? 'nightly' : 'release'/u); + } }); test('the packaged-app probe rejects a mismatched Runtime Host setup package', async () => { @@ -722,13 +914,14 @@ test('one product workflow gates one draft release on every required artifact', for (const verifier of [ 'Verify the final DMG', 'Verify the Windows release', + 'Verify the Linux release', 'Prove deterministic mid-install failure rollback', ]) { const verifierIndex = desktopStepNames.indexOf(verifier); assert.ok(verifierIndex >= 0 && verifierIndex < uploadIndex); } for (const [jobName, group] of [ - ['desktop', 'desktop-${{ matrix.platform }}'], + ['desktop', 'desktop-${{ matrix.platform }}-${{ matrix.arch }}'], ['cli-macos-arm64', 'cli-macos-arm64'], ]) { const stage = jobs[jobName].steps.find( @@ -740,17 +933,78 @@ test('one product workflow gates one draft release on every required artifact', ); assert.equal(upload.with.path, '${{ runner.temp }}/release-assets'); } + // The macOS feed only becomes the one clients read once both runners' copies + // are here, so the merge has to precede the exact-manifest check. + const publishStepNames = jobs.publish.steps.map((step) => step.name); + assert.ok( + publishStepNames.indexOf('Merge the per-architecture update feeds') < + publishStepNames.indexOf('Verify the exact product artifact manifest'), + ); + const mergeFeeds = jobs.publish.steps.find( + (step) => step.name === 'Merge the per-architecture update feeds', + ).run; + assert.match(mergeFeeds, /product-release-artifacts\.mjs merge-feeds release-assets/u); const verifyArtifacts = jobs.publish.steps.find( (step) => step.name === 'Verify the exact product artifact manifest', ).run; assert.match(verifyArtifacts, /product-release-artifacts\.mjs verify release-assets/u); assert.doesNotMatch(verifyArtifacts, /required=\(|Maka-\*|latest\*\.yml/u); + // Finalize publishes and attests these bytes, so the merged, verified + // directory is handed on under a name the publish job's own `release-*` + // download cannot pick the per-architecture feeds back out of. + const assetsUpload = jobs.publish.steps.find( + (step) => step.name === 'Upload the verified release assets', + ); + assert.equal(assetsUpload.with.name, 'product-release-assets-${{ github.run_attempt }}'); + assert.equal(assetsUpload.with.path, 'release-assets'); + assert.equal(assetsUpload.with['if-no-files-found'], 'error'); + assert.equal(assetsUpload.with['retention-days'], 30); + assert.ok( + publishStepNames.indexOf('Verify the exact product artifact manifest') < + publishStepNames.indexOf('Upload the verified release assets'), + ); + const rawDownload = jobs.publish.steps.find((step) => + String(step.uses).startsWith('actions/download-artifact@'), + ); + assert.doesNotMatch( + assetsUpload.with.name, + new RegExp(`^${rawDownload.with.pattern.split('*')[0]}`, 'u'), + ); + // A workflow matrix cannot be generated from the target descriptor, so it is + // held to it here instead of being a second list that can drift. + const matrix = jobs.desktop.strategy.matrix.include; + assert.deepEqual( + matrix.map((entry) => `${entry.platform}-${entry.arch}`).toSorted(), + desktopReleaseTargets('1.2.3', { nightly: false }) + .map((target) => target.name) + .toSorted(), + ); + // The runner image is the workflow's to choose; what it may not do is choose + // one that disagrees with the row it builds. The native Runtime Host peer is + // never cross-built, so every row runs on its own platform and architecture. + for (const { platform, arch, runner } of matrix) { + if (platform === 'macos') { + assert.match(runner, /^macos-/u); + assert.equal(runner.endsWith('-intel'), arch === 'x64', runner); + } else if (platform === 'windows') { + assert.match(runner, /^windows-/u); + } else { + assert.match(runner, /^ubuntu-/u); + assert.equal(runner.endsWith('-arm'), arch === 'arm64', runner); + } + } + const commands = Object.values(jobs) .flatMap((job) => job.steps ?? []) .map((step) => step.run) .filter((run) => typeof run === 'string') .join('\n'); assert.equal((commands.match(/gh release create/gu) ?? []).length, 1); + // Distributable names belong to the target descriptor. The workflow resolves + // them — by discovering the one packaged file, or by handing the architecture + // to a verifier that reads the descriptor — never by spelling one out. + assert.doesNotMatch(commands, /win-x64\.exe|mac-(arm64|x64)\.(zip|dmg)/u); + assert.equal(Object.hasOwn(jobs['release-identity'].outputs, 'exe'), false); assert.equal(jobs.desktop['timeout-minutes'], 75); assert.match(commands, /npm run package:windows-autoupdate-next/u); assert.match(commands, /npm run verify:windows-autoupdate/u); diff --git a/scripts/release-cli-workflow-policy.test.mjs b/scripts/release-cli-workflow-policy.test.mjs index d61004b437..430c837f1f 100644 --- a/scripts/release-cli-workflow-policy.test.mjs +++ b/scripts/release-cli-workflow-policy.test.mjs @@ -252,6 +252,34 @@ test('finalize preserves npm evidence and owns the single product publication bo assert.doesNotMatch(workflow.slice(workflow.indexOf('\n publish:')), /\$\{\{ inputs\./u); }); +test('finalize consumes the normalized release assets the publish job hands off', () => { + // The runner uploads still carry the per-architecture macOS feeds that the + // Release publish job merges into the one feed clients read. Reassembling + // them here would attest and check a set the release never carries, so + // Finalize takes the single artifact holding the verified published bytes. + const finalize = readWorkflow('release-cli-finalize.yml'); + const steps = workflowSteps(finalize); + const download = namedStep(steps, 'Download the exact verified Release run artifacts'); + const [, artifact] = + /\n\s+name: (\S+)-\$\{\{ needs\.inspect\.outputs\.release_run_attempt \}\}/u.exec(download); + assert.doesNotMatch(download, /pattern:|merge-multiple:/u); + assert.match( + readWorkflow('release.yml'), + new RegExp( + `\\n\\s+name: ${artifact}-\\$\\{\\{ github\\.run_attempt \\}\\}\\n\\s+path: release-assets\\n`, + 'u', + ), + ); + + // Everything downstream reads the one directory that download populates. + const attest = steps.find((step) => step.includes('uses: actions/attest@')); + assert.match(attest, /subject-path: \$\{\{ runner\.temp \}\}\/product-release\/\*/u); + const preflight = namedStep(steps, 'Verify the exact publication input'); + assert.match(preflight, /"\$RUNNER_TEMP\/product-release"/u); + const verify = namedStep(steps, 'Verify the issued provenance'); + assert.match(verify, /find "\$RUNNER_TEMP\/product-release"/u); +}); + test('release workflows select npm from the root packageManager authority', () => { for (const name of [ 'cli-package-validation.yml', diff --git a/scripts/verify-linux-harness.test.mjs b/scripts/verify-linux-harness.test.mjs new file mode 100644 index 0000000000..c4b602b118 --- /dev/null +++ b/scripts/verify-linux-harness.test.mjs @@ -0,0 +1,241 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import assert from 'node:assert/strict'; +import { createHash } from 'node:crypto'; +import { mkdir, mkdtemp, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import test, { after, before } from 'node:test'; +import { desktopReleaseTargets } from './desktop-release-targets.mjs'; +import { + mergeDesktopUpdateFeeds, + verifyDesktopUpdateArtifacts, +} from './desktop-update-contract.mjs'; +import { assertElfArchitecture, verifyLinuxRelease } from './verify-linux.mjs'; + +// The two `e_machine` values this project ships, from the ELF specification. +const EM_X86_64 = 0x3e; +const EM_AARCH64 = 0xb7; + +let workingDirectory; + +before(async () => { + workingDirectory = await mkdtemp(join(tmpdir(), 'maka-linux-harness-')); +}); + +after(async () => { + await rm(workingDirectory, { recursive: true, force: true }); +}); + +/** + * A 64-byte ELF header is enough: the architecture assertion reads the magic, + * `EI_DATA` and `e_machine`, and nothing else. Building the bytes here rather + * than checking in a binary keeps the test readable and runs it on any host — + * which is the point, since the packaging it guards only runs on Linux. + */ +async function writeElf(name, machine, { data = 1, magic = '\x7fELF' } = {}) { + const header = Buffer.alloc(64); + header.write(magic, 0, 'latin1'); + header[4] = 2; // EI_CLASS: ELFCLASS64 + header[5] = data; // EI_DATA: 1 little-endian, 2 big-endian + header[6] = 1; // EI_VERSION + header.writeUInt16LE(2, 16); // e_type: ET_EXEC + if (data === 1) header.writeUInt16LE(machine, 18); + else header.writeUInt16BE(machine, 18); + const path = join(workingDirectory, name); + await writeFile(path, header); + return path; +} + +test('the architecture assertion accepts a binary built for the target', async () => { + await assertElfArchitecture(await writeElf('x64.node', EM_X86_64), 'x64'); + await assertElfArchitecture(await writeElf('arm64.node', EM_AARCH64), 'arm64'); +}); + +test('the architecture assertion rejects the other architecture', async () => { + // The failure this exists for: a runner that cross-built the Runtime Host + // peer produces a package that installs and then dies at launch. + const path = await writeElf('wrong.node', EM_X86_64); + await assert.rejects( + () => assertElfArchitecture(path, 'arm64'), + /is built for ELF machine 0x3e, not arm64/u, + ); +}); + +test('the architecture assertion rejects a file that is not ELF', async () => { + const path = join(workingDirectory, 'not-elf.node'); + await writeFile(path, 'this is not a binary at all, it is text\n'); + await assert.rejects(() => assertElfArchitecture(path, 'x64'), /is not an ELF binary/u); +}); + +test('the architecture assertion rejects a truncated header', async () => { + const path = join(workingDirectory, 'truncated.node'); + await writeFile(path, Buffer.from([0x7f, 0x45, 0x4c, 0x46])); + await assert.rejects(() => assertElfArchitecture(path, 'x64'), /is not an ELF binary/u); +}); + +test('the architecture assertion refuses to read a big-endian header', async () => { + // Reading `e_machine` little-endian out of a big-endian file compares + // garbage, so the mismatch has to be named rather than guessed at. + const path = await writeElf('big-endian.node', EM_X86_64, { data: 2 }); + await assert.rejects( + () => assertElfArchitecture(path, 'x64'), + /is not a little-endian ELF binary/u, + ); +}); + +const FEED_VERSION = '9.9.9'; + +/** + * The shape `package:linux` leaves behind: two payloads from two + * electron-builder runs, each run's own single-payload feed, and the merged feed + * `mergeDesktopUpdateFeeds` writes out of the two. The merge is the production + * one, so what is verified below is the merge and the verifier together — a + * hand-written merged document would only ever prove the verifier. The payload + * bytes are written here and the digests taken from them, so a test that drifts + * one field drifts it away from a feed that was otherwise exact. + */ +async function stageLinuxRelease(name, { nightly = false, drift } = {}) { + const target = desktopReleaseTargets(FEED_VERSION, { nightly }).find( + (entry) => entry.name === 'linux-x64', + ); + const directory = join(workingDirectory, name); + await mkdir(directory, { recursive: true }); + const sourcePaths = []; + for (const payload of target.advertised) { + const bytes = Buffer.from(`${payload} payload bytes\n`); + await writeFile(join(directory, payload), bytes); + // Each electron-builder run rewrites the feed knowing only its own payload; + // `package:linux` moves the first aside under this suffix and merges it back. + const file = { + url: payload, + sha512: createHash('sha512').update(bytes).digest('base64'), + size: bytes.length, + }; + const sourcePath = join(directory, `${target.feed}.${sourcePaths.length}`); + await writeFile( + sourcePath, + feedDocument({ + version: FEED_VERSION, + files: [file], + path: file.url, + sha512: file.sha512, + releaseDate: '2026-01-01T00:00:00.000Z', + }), + 'utf8', + ); + sourcePaths.push(sourcePath); + } + const outputPath = join(directory, target.feed); + const merged = await mergeDesktopUpdateFeeds({ sourcePaths, outputPath }); + await Promise.all(sourcePaths.map((path) => rm(path))); + // Drift is applied to what the merge produced, so each rejection below names a + // field of a real merged document rather than of a fabricated one. + if (drift) { + await writeFile(outputPath, feedDocument(drift(merged)), 'utf8'); + } + return { directory, target }; +} + +/** electron-builder's feed layout, written by hand so this suite stays on node builtins. */ +function feedDocument(feed) { + const lines = [`version: ${feed.version}`, 'files:']; + for (const file of feed.files) { + lines.push(` - url: ${file.url}`, ` sha512: ${file.sha512}`, ` size: ${file.size}`); + } + lines.push(`path: ${feed.path}`, `sha512: ${feed.sha512}`, `releaseDate: '${feed.releaseDate}'`); + return `${lines.join('\n')}\n`; +} + +function verifyStagedFeed({ directory, target }) { + return verifyDesktopUpdateArtifacts({ + directory, + metadataName: target.feed, + version: FEED_VERSION, + artifactNames: target.advertised, + }); +} + +test('the merged Linux feed offers both distributables', async () => { + // The AppImage and the deb are built by separate electron-builder runs, the + // second of which overwrites the first's feed; only the merge puts both in + // one document, and until this ran nothing opened the merged bytes before + // publication. + const staged = await stageLinuxRelease('merged'); + const { metadata } = await verifyStagedFeed(staged); + assert.deepEqual( + metadata.files.map((file) => file.url).sort(), + [...staged.target.advertised].sort(), + ); +}); + +test('the Nightly merged feed is read under its own name', async () => { + // The Nightly publishes the same two payloads through `dev-linux.yml`, so a + // verifier that only ever looked for `latest-linux.yml` would check nothing + // there. + const staged = await stageLinuxRelease('merged-nightly', { nightly: true }); + assert.equal(staged.target.feed, 'dev-linux.yml'); + await verifyStagedFeed(staged); +}); + +test('a merged feed that lost a payload is rejected', async () => { + // The exact damage the merge can do: the deb's own feed contributes nothing + // and the AppImage's document survives alone, offering deb users no update. + const staged = await stageLinuxRelease('dropped-payload', { + drift: (feed) => ({ ...feed, files: feed.files.slice(0, 1) }), + }); + await assert.rejects(() => verifyStagedFeed(staged), /advertises \[/u); +}); + +test('a merged feed whose digest belongs to another payload is rejected', async () => { + // `path` and the top-level `sha512` are the primary payload's identity; the + // merge keeps the first document's, so pointing `path` at the other file + // leaves an updater verifying the deb against the AppImage's digest. + const staged = await stageLinuxRelease('mismatched-primary', { + drift: (feed) => ({ ...feed, path: feed.files[1].url }), + }); + await assert.rejects(() => verifyStagedFeed(staged), /inconsistent payload identity/u); +}); + +test('a merged feed carrying a stale digest is rejected', async () => { + const staged = await stageLinuxRelease('stale-digest', { + drift: (feed) => ({ + ...feed, + files: [feed.files[0], { ...feed.files[1], sha512: createHash('sha512').digest('base64') }], + }), + }); + await assert.rejects(() => verifyStagedFeed(staged), /sha512 does not match/u); +}); + +test('a merged feed carrying a stale size is rejected', async () => { + const staged = await stageLinuxRelease('stale-size', { + drift: (feed) => ({ + ...feed, + files: [feed.files[0], { ...feed.files[1], size: feed.files[1].size + 1 }], + }), + }); + await assert.rejects(() => verifyStagedFeed(staged), /records .* size/u); +}); + +test('Linux verification refuses to run anywhere else', async () => { + // The distributables only exist on the runner that built them, so running + // this elsewhere would otherwise fail later and less clearly. + await assert.rejects(() => verifyLinuxRelease('x64', { platform: 'darwin' }), /requires Linux/u); +}); diff --git a/scripts/verify-linux.mjs b/scripts/verify-linux.mjs new file mode 100644 index 0000000000..77129fcad2 --- /dev/null +++ b/scripts/verify-linux.mjs @@ -0,0 +1,240 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import { access, chmod, mkdtemp, open, readdir, readFile, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { basename, dirname, join, resolve } from 'node:path'; +import { fileURLToPath, pathToFileURL } from 'node:url'; +import { resolveDesktopReleaseTarget } from './desktop-nightly.mjs'; +import { + assertPackagedUpdateConfiguration, + verifyDesktopUpdateArtifacts, +} from './desktop-update-contract.mjs'; +import { + assertMissing, + assertPackagedDependencyClosure, + assertPackagedResources, + runCommand, + sha256File, + smokePackagedRenderer, +} from './verify-packaged-app.mjs'; + +const repoRoot = dirname(dirname(fileURLToPath(import.meta.url))); + +/** + * `e_machine`, at offset 18 of every ELF header. This is the one field that + * says which processor the file was built for. + */ +const ELF_MACHINES = Object.freeze({ x64: 0x3e, arm64: 0xb7 }); + +/** + * Debian policy allows only lowercase letters, digits and `-+.` in a package + * name, and `dpkg` refuses to install one that breaks the rule. The name is + * derived from the product name rather than configured, so nothing else in + * this repository would notice a capital letter reaching it. + */ +const DEBIAN_PACKAGE_NAME = /^[a-z0-9][a-z0-9+.-]+$/u; + +function runCommandFromRepo(command, args, options = {}) { + return runCommand(command, args, { cwd: repoRoot, ...options }); +} + +/** + * The packaging scripts refuse to cross-build because the Runtime Host peer is + * a host binary, and a runner that shipped the other architecture's peer would + * produce a package that installs and then fails at launch. Only reading the + * ELF header proves which one is actually inside. + */ +export async function assertElfArchitecture(path, arch) { + const handle = await open(path); + let header; + try { + header = Buffer.alloc(20); + const { bytesRead } = await handle.read(header, 0, 20, 0); + if (bytesRead < 20 || header.toString('latin1', 0, 4) !== '\x7fELF') { + throw new Error(`${basename(path)} is not an ELF binary`); + } + } finally { + await handle.close(); + } + // EI_DATA: every architecture this project builds is little-endian, and + // reading `e_machine` the wrong way round would silently compare garbage. + if (header[5] !== 1) { + throw new Error(`${basename(path)} is not a little-endian ELF binary`); + } + const machine = header.readUInt16LE(18); + if (machine !== ELF_MACHINES[arch]) { + throw new Error( + `${basename(path)} is built for ELF machine 0x${machine.toString(16)}, not ${arch}`, + ); + } +} + +/** + * Where fpm placed the application is discovered from the extracted tree rather + * than derived from electron-builder's install prefix and product name. Deriving + * what a payload should contain, instead of reading what it does, is exactly how + * this verifier came to hand a checksum to a file it had never opened. Exactly + * one match is required: `find` would silently pick either of two. + */ +async function debResourcesDirectory(root) { + const suffix = join('resources', 'app.asar'); + const entries = await readdir(root, { recursive: true }); + const matches = entries.filter((entry) => entry.endsWith(suffix)); + if (matches.length !== 1) { + throw new Error(`The deb contains ${matches.length} resources/app.asar entries, expected 1`); + } + return join(root, dirname(matches[0])); +} + +/** + * Both Linux distributables are verified here, and each is opened. They are + * built by two separate electron-builder runs — the split is what keeps the + * deb's `package-type` marker out of the AppImage — so nothing proven about one + * carries over to the other. `--appimage-extract` is handled by the AppImage + * runtime itself and needs no FUSE mount; `dpkg-deb` ships with the runner. + * + * The renderer smoke test needs a display, so the caller runs this whole script + * under `xvfb-run`. It is applied to the AppImage only: extracting it produces + * the same tree its runtime mounts at launch, so running that tree is running + * the artifact. A deb extracted with `dpkg-deb -x` is not an installation — + * `dpkg` would still have to set the sandbox helper's setuid bit — so launching + * it would prove something about a tree no user ever has. + */ +export async function verifyLinuxRelease( + arch, + { + platform = process.platform, + run = runCommandFromRepo, + requirePath = access, + forbidPath = assertMissing, + environment = process.env, + checksum = sha256File, + smokeRenderer = smokePackagedRenderer, + assertArchitecture = assertElfArchitecture, + } = {}, +) { + if (platform !== 'linux') { + throw new Error('Linux release verification requires Linux.'); + } + + // The AppImage and the deb never share a spelling of the architecture, so the + // target descriptor is the only place that knows both names. + const target = await resolveDesktopReleaseTarget(`linux-${arch}`, { environment }); + const appImagePath = resolve(target.payloadPath('.AppImage')); + const debPath = resolve(target.payloadPath('.deb')); + await access(appImagePath); + await access(debPath); + // The descriptor already resolved and validated the channel when it resolved + // the version; re-reading the environment here would let an unvalidated value + // name one channel while the artifacts it just named came from the other. + const channel = target.nightly ? 'nightly' : 'release'; + const workingDirectory = await mkdtemp(join(tmpdir(), 'maka-release-verify-')); + const peerBinary = join('runtime-host-peer', 'maka_runtime_host_peer.node'); + + try { + await chmod(appImagePath, 0o755); + await run(appImagePath, ['--appimage-extract'], { cwd: workingDirectory }); + const squashfsRoot = join(workingDirectory, 'squashfs-root'); + const appImageResources = join(squashfsRoot, 'resources'); + + await assertPackagedResources(appImageResources, { requirePath, forbidPath }); + // The deb target writes this marker into the shared unpacked tree, and it is + // what electron-updater reads to pick DebUpdater over AppImageUpdater. An + // AppImage carrying it would try to update itself by installing a deb. + await forbidPath(join(appImageResources, 'package-type')); + await assertPackagedUpdateConfiguration(appImageResources, { channel }); + await assertPackagedDependencyClosure(appImageResources); + await assertArchitecture(join(appImageResources, peerBinary), arch); + + const debRoot = join(workingDirectory, 'deb'); + await run('dpkg-deb', ['-x', debPath, debRoot]); + const debResources = await debResourcesDirectory(debRoot); + + await assertPackagedResources(debResources, { requirePath, forbidPath }); + await assertPackagedUpdateConfiguration(debResources, { channel }); + await assertPackagedDependencyClosure(debResources); + await assertArchitecture(join(debResources, peerBinary), arch); + // The mirror of the AppImage assertion above. This marker is what sends the + // packaged updater down DebUpdater, and the deb is the one payload that has + // to carry it: without it an installed deb would try to update itself by + // replacing an AppImage that is not there. + const packageType = (await readFile(join(debResources, 'package-type'), 'utf8')).trim(); + if (packageType !== 'deb') { + throw new Error(`The deb declares package-type ${packageType || '(empty)'}`); + } + // electron-builder builds this name out of the product name, so nothing + // else here would catch a capital letter reaching `dpkg`, which rejects it. + const { stdout: declaredName } = await run('dpkg-deb', ['-f', debPath, 'Package']); + if (!DEBIAN_PACKAGE_NAME.test(declaredName.trim())) { + throw new Error(`The deb declares an uninstallable package name: ${declaredName.trim()}`); + } + // fpm records the architecture it was told to build; the descriptor names + // the file after the architecture it asked for. A runner that produced the + // wrong one would otherwise publish it under the right name. + const namedArchitecture = /-([^-]+)\.deb$/u.exec(basename(debPath))?.[1]; + const { stdout } = await run('dpkg-deb', ['-f', debPath, 'Architecture']); + if (stdout.trim() !== namedArchitecture) { + throw new Error(`${basename(debPath)} contains architecture ${stdout.trim() || '(none)'}`); + } + + // Every assertion above reads files. This one runs the application, the way + // the macOS and Windows verifications already do, and is the only thing here + // that can fail on a package that is structurally perfect and still cannot + // start — a missing shared library, or a sandbox the host will not grant. + await requirePath(join(squashfsRoot, 'AppRun')); + await smokeRenderer(join(squashfsRoot, 'AppRun'), { workingDirectory }); + + // Linux is the one platform whose update feed this repository assembles + // itself: `package:linux` runs electron-builder twice and merges the two + // feeds, so the merged bytes are the only ones no build step ever wrote as + // a whole. Reading them here, against the payloads just verified, is what + // keeps a dropped entry or a stale digest from surviving until publication. + await verifyDesktopUpdateArtifacts({ + directory: target.releaseDirectory, + metadataName: target.feed, + version: target.version, + artifactNames: target.advertised, + }); + + // A formal release publishes a checksum beside each distributable, the way + // the Windows verification does for its installer and archive. Each one is + // issued only for a payload every assertion above has already accepted. + const checksums = []; + for (const path of target.checksumPaths()) { + const sha256 = await checksum(path); + const checksumPath = `${path}.sha256`; + await writeFile(checksumPath, `${sha256} ${basename(path)}\n`, 'utf8'); + checksums.push({ path, checksumPath, sha256 }); + } + + return { appImagePath, debPath, checksums }; + } finally { + await rm(workingDirectory, { recursive: true, force: true }); + } +} + +if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) { + const result = await verifyLinuxRelease(process.argv[2] ?? process.arch); + console.log(`Verified ${result.appImagePath}`); + console.log(`Verified ${result.debPath}`); + for (const { path, sha256 } of result.checksums) { + console.log(`SHA-256 ${sha256} ${basename(path)}`); + } +} diff --git a/scripts/verify-macos-autoupdate.mjs b/scripts/verify-macos-autoupdate.mjs index 4a548bb790..2a3c39b620 100644 --- a/scripts/verify-macos-autoupdate.mjs +++ b/scripts/verify-macos-autoupdate.mjs @@ -30,6 +30,7 @@ import { startDesktopUpdateFeed, verifyDesktopUpdateArtifacts, } from './desktop-update-contract.mjs'; +import { resolveDesktopReleaseTarget } from './desktop-nightly.mjs'; import { evaluateInRenderer, findRendererTarget, @@ -120,11 +121,11 @@ async function stopProcess(processId) { /** check → download → Squirrel.Mac replacement → automatic relaunch → smoke. */ export async function verifyMacosAutoupdate( - candidateInput, nextDirectoryInput, { platform = process.platform, arch = process.arch, + environment = process.env, run = runCommand, makeTemporaryDirectory = () => mkdtemp(join(tmpdir(), 'maka-macos-autoupdate-')), smokeRenderer = smokePackagedRenderer, @@ -133,13 +134,15 @@ export async function verifyMacosAutoupdate( if (platform !== 'darwin' || arch !== 'arm64') { throw new Error('macOS auto-update verification requires an Apple Silicon macOS host.'); } - if (!candidateInput || !nextDirectoryInput) { - throw new Error( - 'Usage: npm run verify:macos-autoupdate -- ', - ); + if (!nextDirectoryInput) { + throw new Error('Usage: npm run verify:macos-autoupdate -- '); } - const candidateZip = resolve(candidateInput); + // The candidate is the archive this runner just packaged, so the descriptor + // names it here the way `verify:macos` names its DMG, rather than the workflow + // spelling it out a second time. + const candidate = await resolveDesktopReleaseTarget(`macos-${arch}`, { environment }); + const candidateZip = resolve(candidate.payloadPath('.zip')); const nextDirectory = resolve(nextDirectoryInput); await access(candidateZip); const metadata = parseYaml(await readFile(join(nextDirectory, 'latest-mac.yml'), 'utf8')); @@ -152,7 +155,7 @@ export async function verifyMacosAutoupdate( directory: nextDirectory, metadataName: 'latest-mac.yml', version: nextVersion, - artifactName: nextZipName, + artifactNames: [nextZipName], }); const temporaryDirectory = await makeTemporaryDirectory(); @@ -326,5 +329,5 @@ export async function verifyMacosAutoupdate( } if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) { - await verifyMacosAutoupdate(process.argv[2], process.argv[3]); + await verifyMacosAutoupdate(process.argv[2]); } diff --git a/scripts/verify-macos-arm64-dmg.mjs b/scripts/verify-macos-dmg.mjs similarity index 74% rename from scripts/verify-macos-arm64-dmg.mjs rename to scripts/verify-macos-dmg.mjs index 1d9fa1f1ac..a8ea3ee971 100644 --- a/scripts/verify-macos-arm64-dmg.mjs +++ b/scripts/verify-macos-dmg.mjs @@ -33,7 +33,11 @@ import { fileURLToPath, pathToFileURL } from 'node:url'; import { FILESYSTEM_WORKER_PROTOCOL_VERSION } from '../packages/runtime/dist/filesystem-worker/protocol.js'; import { readProductManifestIdentity } from './product-release-identity.mjs'; import { assertPackagedUpdateConfiguration } from './desktop-update-contract.mjs'; -import { resolveDesktopBuildVersion, resolveRuntimeHostSetupPackage } from './desktop-nightly.mjs'; +import { + resolveDesktopBuildVersion, + resolveDesktopReleaseTarget, + resolveRuntimeHostSetupPackage, +} from './desktop-nightly.mjs'; import { assertMissing, assertPackagedDependencyClosure, @@ -106,10 +110,12 @@ export async function smokePackagedFilesystemWorker( } } -function assertSingleArchitecture(output, subject) { +function assertSingleArchitecture(output, subject, expectedArch) { const architectures = output.trim().split(/\s+/).filter(Boolean); - if (architectures.length !== 1 || architectures[0] !== 'arm64') { - throw new Error(`${subject} must contain only arm64, found: ${architectures.join(', ')}`); + if (architectures.length !== 1 || architectures[0] !== expectedArch) { + throw new Error( + `${subject} must contain only ${expectedArch}, found: ${architectures.join(', ')}`, + ); } } @@ -128,6 +134,14 @@ export async function verifyPackagedMacApp( smokeFilesystemWorker = smokePackagedFilesystemWorker, workingDirectory = dirname(appPath), environment = process.env, + // The architecture the DMG was chosen for, not the one this process happens + // to run as: `process.arch` under Rosetta, or on a runner packaging the + // other slice, would assert the wrong thing about the binary it opened. + expectedArch = process.arch, + // Which channel the packaged client points at is the descriptor's to decide, + // so the caller that resolved the target passes it. Nothing but the release + // verifier can see a nightly descriptor. + channel = 'release', } = {}, ) { const product = await readProductManifestIdentity(); @@ -151,13 +165,11 @@ export async function verifyPackagedMacApp( await requirePath(executable); await assertPackagedResources(resources, { requirePath, forbidPath }); - await assertPackagedUpdateConfiguration(resources, { - channel: environment.MAKA_DESKTOP_NIGHTLY_VERSION ? 'nightly' : 'release', - }); + await assertPackagedUpdateConfiguration(resources, { channel }); await assertPackagedDependencyClosure(resources); const executableArchitectures = await run('lipo', ['-archs', executable]); - assertSingleArchitecture(executableArchitectures.stdout, 'Maka executable'); + assertSingleArchitecture(executableArchitectures.stdout, 'Maka executable', expectedArch); await run('codesign', ['--verify', '--deep', '--strict', '--verbose=2', appPath]); await run('spctl', ['--assess', '--type', 'execute', '--verbose=4', appPath]); await run('xcrun', ['stapler', 'validate', appPath]); @@ -177,18 +189,27 @@ export async function verifyPackagedMacApp( await smokeRenderer(executable, { workingDirectory }); } -export async function verifyMacosArm64Dmg( - inputPath, - { platform = process.platform, run = runCommandFromRepo, verifyApp = verifyPackagedMacApp } = {}, +export async function verifyMacosDmg( + arch, + { + platform = process.platform, + run = runCommandFromRepo, + verifyApp = verifyPackagedMacApp, + environment = process.env, + checksum = sha256File, + } = {}, ) { if (platform !== 'darwin') { throw new Error('DMG verification requires macOS.'); } - if (!inputPath) { - throw new Error('Usage: npm run verify:macos-arm64 -- '); - } - const dmgPath = resolve(inputPath); + // Named from the descriptor rather than handed in as a path, the way + // `verify:linux` already resolves its own payloads. The workflows used to + // spell this name out in YAML, which put a second authority on the artifact + // name beside the descriptor — and, unlike it, that copy was checked by + // nothing. + const target = await resolveDesktopReleaseTarget(`macos-${arch}`, { environment }); + const dmgPath = resolve(target.payloadPath('.dmg')); await access(dmgPath); const temporaryDirectory = await mkdtemp(join(tmpdir(), 'maka-release-verify-')); const mountpoint = join(temporaryDirectory, 'mounted'); @@ -211,18 +232,30 @@ export async function verifyMacosArm64Dmg( } try { - await verifyApp(copiedApp, { workingDirectory: temporaryDirectory }); - const sha256 = await sha256File(dmgPath); - const checksumPath = `${dmgPath}.sha256`; - await writeFile(checksumPath, `${sha256} ${basename(dmgPath)}\n`, 'utf8'); - return { dmgPath, checksumPath, sha256 }; + await verifyApp(copiedApp, { + workingDirectory: temporaryDirectory, + expectedArch: arch, + channel: target.nightly ? 'nightly' : 'release', + }); + // Which payloads a formal release publishes a `.sha256` beside is the + // descriptor's to decide, the way `verify:linux` already reads it. + const checksums = []; + for (const path of target.checksumPaths()) { + const sha256 = await checksum(path); + const checksumPath = `${path}.sha256`; + await writeFile(checksumPath, `${sha256} ${basename(path)}\n`, 'utf8'); + checksums.push({ path, checksumPath, sha256 }); + } + return { dmgPath, checksums }; } finally { await rm(temporaryDirectory, { recursive: true, force: true }); } } if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) { - const result = await verifyMacosArm64Dmg(process.argv[2]); + const result = await verifyMacosDmg(process.argv[2] ?? process.arch); console.log(`Verified ${result.dmgPath}`); - console.log(`SHA-256 ${result.sha256}`); + for (const { path, sha256 } of result.checksums) { + console.log(`SHA-256 ${sha256} ${basename(path)}`); + } } diff --git a/scripts/verify-windows-autoupdate.mjs b/scripts/verify-windows-autoupdate.mjs index fe0a1f477d..eccd7eda81 100644 --- a/scripts/verify-windows-autoupdate.mjs +++ b/scripts/verify-windows-autoupdate.mjs @@ -23,6 +23,7 @@ import { access, mkdir, mkdtemp, readFile, rm, stat } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { basename, join, resolve } from 'node:path'; import { pathToFileURL } from 'node:url'; +import { desktopReleaseTargets } from './desktop-release-targets.mjs'; import { feedServed, startDesktopUpdateFeed, @@ -140,13 +141,17 @@ export async function verifyWindowsAutoupdate( `The served version ${nextVersion} must be newer than the candidate ${candidateVersion}.`, ); } - const nextInstallerName = `Maka-${nextVersion}-win-x64.exe`; + // The served build is the same target under a bumped version, so the + // descriptor names its installer here too. + const nextInstallerName = desktopReleaseTargets(nextVersion, { nightly: false }) + .find((entry) => entry.name === 'windows-x64') + .payloads.find((name) => name.endsWith('.exe')); installerVersion(join(nextDirectory, nextInstallerName)); await verifyDesktopUpdateArtifacts({ directory: nextDirectory, metadataName: 'latest.yml', version: nextVersion, - artifactName: nextInstallerName, + artifactNames: [nextInstallerName], }); const temporaryDirectory = await makeTemporaryDirectory(); diff --git a/scripts/verify-windows-x64.mjs b/scripts/verify-windows-x64.mjs index 3b19751c04..9f14c0b333 100644 --- a/scripts/verify-windows-x64.mjs +++ b/scripts/verify-windows-x64.mjs @@ -24,7 +24,11 @@ import { basename, dirname, join, resolve } from 'node:path'; import { fileURLToPath, pathToFileURL } from 'node:url'; import { readProductManifestIdentity } from './product-release-identity.mjs'; import { assertPackagedUpdateConfiguration } from './desktop-update-contract.mjs'; -import { resolveDesktopBuildVersion, resolveRuntimeHostSetupPackage } from './desktop-nightly.mjs'; +import { + resolveDesktopBuildVersion, + resolveDesktopReleaseTarget, + resolveRuntimeHostSetupPackage, +} from './desktop-nightly.mjs'; import { assertMissing, assertPackagedDependencyClosure, @@ -127,6 +131,11 @@ export async function verifyPackagedWindowsApp( expectedVersion, artifactContract = 'current', environment = process.env, + // Which channel the packaged client points at is the descriptor's to decide, + // so the caller that resolved the target passes it. The installer-lifecycle + // and autoupdate verifications run on the formal release lanes alone, which + // have no nightly descriptor to resolve. + channel = 'release', } = {}, ) { if (artifactContract !== 'current' && artifactContract !== 'upgrade-baseline') { @@ -154,9 +163,7 @@ export async function verifyPackagedWindowsApp( // own commit: its update feed and dependency closure are the ones that were // right for it, not the ones this checkout expects. if (requiresCurrentContract) { - await assertPackagedUpdateConfiguration(resources, { - channel: environment.MAKA_DESKTOP_NIGHTLY_VERSION ? 'nightly' : 'release', - }); + await assertPackagedUpdateConfiguration(resources, { channel }); await assertPackagedDependencyClosure(resources); } @@ -284,21 +291,26 @@ export async function verifyPackagedWindowsApp( // checklist step. (macOS mounts its DMG instead because notarizing and stapling // rewrite the DMG after packaging, so only the final artifact can be trusted.) export async function verifyWindowsX64Release( - inputPath, - { platform = process.platform, verifyApp = verifyPackagedWindowsApp, checksum = sha256File } = {}, + arch, + { + platform = process.platform, + verifyApp = verifyPackagedWindowsApp, + checksum = sha256File, + environment = process.env, + } = {}, ) { if (platform !== 'win32') { throw new Error('Windows release verification requires Windows.'); } - if (!inputPath) { - throw new Error('Usage: npm run verify:windows-x64 -- '); - } - const exePath = resolve(inputPath); - if (!exePath.endsWith('.exe')) { - throw new Error(`Expected the NSIS installer .exe, found ${basename(exePath)}.`); - } - const zipPath = `${exePath.slice(0, -'.exe'.length)}.zip`; + // Named from the descriptor rather than handed in as a path, the way + // `verify:linux` and `verify:macos` already resolve their own payloads. The + // workflows used to spell the installer name out in YAML, which put a second + // authority on the artifact name beside the descriptor — and, unlike it, that + // copy was checked by nothing. + const target = await resolveDesktopReleaseTarget(`windows-${arch}`, { environment }); + const exePath = resolve(target.payloadPath('.exe')); + const zipPath = resolve(target.payloadPath('.zip')); const unpackedDirectory = join(dirname(exePath), 'win-unpacked'); await access(exePath); await access(zipPath); @@ -309,11 +321,16 @@ export async function verifyWindowsX64Release( const temporaryDirectory = await mkdtemp(join(tmpdir(), 'maka-release-verify-')); try { - await verifyApp(unpackedDirectory, { workingDirectory: temporaryDirectory }); + await verifyApp(unpackedDirectory, { + workingDirectory: temporaryDirectory, + channel: target.nightly ? 'nightly' : 'release', + }); step('checksumming the release artifacts'); + // Which payloads a formal release publishes a `.sha256` beside is the + // descriptor's to decide, the way `verify:linux` already reads it. const checksums = []; - for (const path of [exePath, zipPath]) { + for (const path of target.checksumPaths()) { const sha256 = await checksum(path); const checksumPath = `${path}.sha256`; await writeFile(checksumPath, `${sha256} ${basename(path)}\n`, 'utf8'); @@ -336,7 +353,7 @@ export async function verifyWindowsX64Release( } if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) { - const result = await verifyWindowsX64Release(process.argv[2]); + const result = await verifyWindowsX64Release(process.argv[2] ?? process.arch); console.log(`Verified ${result.exePath}`); for (const { path, sha256 } of result.checksums) { console.log(`SHA-256 ${sha256} ${basename(path)}`);