Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 99 additions & 14 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -1,18 +1,14 @@
name: Release

# Package + publish on a version tag. The matrix lives here (the caller owns it);
# each leg calls the single-combo reusable helper. The publish feed is derived from
# distro-release/distro-channel. The tag must equal the avocado.yaml version.
# Build the agent binaries, attach them to the release, then package + publish the
# extension. The tag must equal the avocado.yaml version.
#
# The matrix dimension is the FEED, not the target. An extension version is
# global within a (distro-release, distro-channel), so two rows differing only by
# target publish the same version twice into the same feed and the second fails
# with "that extension version is already taken". Connect fans a published
# extension out to the per-target feeds itself - avocado-ext-cli and
# avocado-ext-microclaw are arch-specific and reach both qemux86-64-ext and
# imx93-frdm-ext while publishing only a qemux86-64 row - so `target` here just
# names the target used to package, and adding rows for other targets buys
# nothing while breaking the release.
# Two matrices here answer different questions, and conflating them broke 0.1.0.
# `binaries` varies ARCHITECTURE, because a binary is architecture-specific.
# `publish` varies FEED, because an extension version is global within a distro
# release and channel - two publish rows differing only by target send the same
# version into the same feed and the second is refused as a duplicate. Connect
# distributes a published extension to the per-target feeds itself.
on:
push:
tags:
Expand All @@ -21,10 +17,99 @@ on:
workflow_dispatch:

jobs:
binaries:
name: Build ${{ matrix.rust-target }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- { rust-target: x86_64-unknown-linux-musl }
- { rust-target: aarch64-unknown-linux-musl }
steps:
- uses: actions/checkout@v5

- name: Install cross
run: cargo install cross --locked --version 0.2.5

# musl, statically linked, so the artifact carries no libc version coupling
# and runs on any Avocado target of this architecture whatever the device's
# glibc is. A gnu build would pin the runner's glibc into the binary and
# fail on older devices at exec time rather than here.
- name: Build
working-directory: agent
run: cross build --release --target ${{ matrix.rust-target }}

- name: Verify the artifact is static and the right architecture
working-directory: agent
run: |
bin="target/${{ matrix.rust-target }}/release/avocado-container-agent-dev"
desc="$(file -b "$bin")"
echo "$desc"
case "$desc" in
*"statically linked"*|*"static-pie linked"*) ;;
*) echo "::error::artifact is not static: $desc" >&2; exit 1 ;;
esac
case "${{ matrix.rust-target }}" in
x86_64-*) want="x86-64" ;;
aarch64-*) want="aarch64" ;;
esac
case "$desc" in
*"$want"*) ;;
*) echo "::error::artifact is not $want: $desc" >&2; exit 1 ;;
esac

- name: Package
working-directory: agent
run: |
asset="avocado-container-agent-dev-${GITHUB_REF_NAME}-${{ matrix.rust-target }}.tar.gz"
tar -czf "../$asset" -C "target/${{ matrix.rust-target }}/release" \
avocado-container-agent-dev
echo "asset=$asset" >> "$GITHUB_ENV"

- uses: actions/upload-artifact@v4
with:
name: ${{ matrix.rust-target }}
path: ${{ env.asset }}
if-no-files-found: error

release-assets:
name: Attach binaries to the release
needs: binaries
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/download-artifact@v4
with:
path: dist
merge-multiple: true

# cad-compile.sh refuses when SHA256SUMS does not list the asset it wants,
# so an architecture missing here fails the consumer's build loudly rather
# than leaving the agent silently absent from their image.
- name: Checksum
working-directory: dist
run: |
sha256sum *.tar.gz > SHA256SUMS
cat SHA256SUMS

- name: Upload to the release
working-directory: dist
env:
GH_TOKEN: ${{ github.token }}
run: |
gh release create "$GITHUB_REF_NAME" --repo "$GITHUB_REPOSITORY" \
--title "$GITHUB_REF_NAME" --notes "Agent binaries for $GITHUB_REF_NAME." \
|| echo "release already exists, uploading into it"
gh release upload "$GITHUB_REF_NAME" --repo "$GITHUB_REPOSITORY" \
--clobber *.tar.gz SHA256SUMS

publish:
# Gated on the assets existing: a consumer's build fetches them, so an
# extension published ahead of its binaries is one nobody can build against.
needs: release-assets
strategy:
# Each feed reports its own outcome. Without this, one feed's failure
# cancels the other and a partial publish looks like a total one.
fail-fast: false
matrix:
include:
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Cargo build output for the staged crate.
/agent/target/
/agent/target*/

# Cross-compilation config written by cad-compile.sh at build time. It pins one
# machine's SDK sysroot path, so a committed copy is wrong for everyone else.
Expand Down
25 changes: 25 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,31 @@ All notable changes to avocado-ext-container-agent-dev are documented in this fi
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.2.0]

### Changed
- The extension ships a prebuilt agent binary instead of its source. `avocado
ext package` runs no build step, so the published RPM could only carry what
was in the repo; shipping the crate pushed the compile onto consumers, who
have no Rust cross toolchain in their SDK and no reason to. A clean project
following the documented setup failed inside `ring`'s build script with
`ToolNotFound: failed to find tool "x86_64-avocado-linux-gcc"`, after both
`avocado install` and the extension's own sysroot creation reported success.
- Binaries are statically linked against musl, so they carry no libc version
coupling and run on any Avocado target of the right architecture.
- `release.yml` builds x86_64 and aarch64 binaries, verifies each is static and
the right architecture, and attaches them with a `SHA256SUMS` to the release
before the extension publishes.
- `cad-install.sh` checks the staged binary's architecture against the target
before installing. The packaged extension is noarch and nothing downstream
re-checks, so a mismatch would otherwise install cleanly and fail at exec time
on the device.

### Removed
- The Rust cross toolchain from the extension's SDK requirements, and `libstd-rs`
from the compile section. Nothing compiles at build time any more.
- The crate source from `package_files`. The RPM is no longer a source package.

## [0.1.1]

### Fixed
Expand Down
30 changes: 14 additions & 16 deletions avocado.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ supported_targets:

extensions:
avocado-ext-container-agent-dev:
version: 0.1.1
version: 0.2.0
release: r0
summary: Container Dev Mode device agent (development only)
description: |
Expand Down Expand Up @@ -40,10 +40,12 @@ extensions:
url: https://github.com/avocado-linux/ext-container-agent-dev
vendor: Avocado Linux <info@avocadolinux.org>

# The crate is staged and cross-compiled at package time, so its sources are
# part of the payload rather than a prebuilt artifact.
# No `agent` here. The crate is no longer part of the payload: cad-compile.sh
# fetches a prebuilt static binary from this version's release instead of
# compiling. Shipping the source made the RPM a source package, which pushed
# the compile onto consumers who have no Rust cross toolchain in their SDK.
# avocado.yaml stays because cad-compile.sh reads the version out of it.
package_files:
- agent
- overlay
- avocado.yaml
- cad-*.sh
Expand Down Expand Up @@ -72,15 +74,12 @@ extensions:
compile: avocado-container-agent-dev
install: cad-install.sh

sdk:
packages:
nativesdk-binutils: '*'
nativesdk-cargo: '*'
nativesdk-gcc: '*'
nativesdk-glibc-dev: '*'
nativesdk-libgcc-dev: '*'
nativesdk-rust: '*'
packagegroup-rust-cross-canadian-avocado-{{ avocado.target }}: '*'
# No SDK toolchain. Nothing is compiled at build time any more, so a consumer
# needs no Rust cross toolchain in their SDK - which is the point of the
# switch. These declarations did not reach a consumer anyway: a package-sourced
# extension does not carry its sdk.packages across, so the build failed inside
# ring's build script on a machine that had never been told it needed a
# compiler.

# Release-scoped, not channel-scoped: this resolves to avocadolinux/sdk:2026.
# The SDK image is per-release only - appending the channel points at a tag that
Expand All @@ -93,8 +92,7 @@ sdk:
avocado-container-agent-dev:
compile: cad-compile.sh
clean: cad-clean.sh
packages:
libstd-rs: '*'
libstd-rs-dev: '*'
# No libstd-rs. The hook fetches and unpacks a prebuilt binary; it runs no
# compiler, so the Rust runtime libraries it used to need are dead weight.
packages:
avocado-sdk-toolchain: '*'
156 changes: 69 additions & 87 deletions cad-compile.sh
Original file line number Diff line number Diff line change
@@ -1,103 +1,85 @@
#!/bin/bash
set -euo pipefail

RUST_TARGET=""
match_count=0
matches=""

# Resolve the Rust target from RUST_TARGET_PATH. Enumerate every candidate
# instead of stopping at the first: an OECORE_TARGET_ARCH that prefix-matches
# two target JSONs used to take whichever the glob happened to return first and
# discard the rest without a word. That is the worst failure shape available
# here - a binary for the wrong architecture packages, publishes and installs
# successfully, and only fails when the device tries to exec it, several layers
# from the cause. Refusing to guess keeps the failure local and attributable.
# Stage the prebuilt agent binary for this target so cad-install.sh can place it.
#
# devtool-debt: this block is duplicated, still unfixed, in the three sibling
# extensions that have not yet moved to their own ext-* repo - cli, connect and
# tunnels - in both their compile and install scripts. jtop and microclaw carry
# a copy too, but theirs are superseded: both migrated to ext-* repos, and
# ext-microclaw resolves the architecture with a plain case statement that does
# not have this defect at all.
# Ceiling: those three are safe by GLOB ORDERING ALONE, not by construction.
# Every target whose architecture matches the SDK host architecture has two
# matching JSONs - the device triple and the SDK's own <arch>-avocadosdk-* - and
# first-match-and-break survives only because "avocado-" sorts ahead of
# "avocadosdk-" on the hyphen. Nothing enforces that ordering. This is not a
# theoretical ceiling: it is exactly what broke qemux86-64 here the first time
# the guard below ran.
# Upgrade trigger: a vendor string that reorders those two, a third triple
# sorting between them, or any of the three gaining a target whose architecture
# prefix-matches more than one device triple.
for json_file in "$RUST_TARGET_PATH"/*.json; do
if [ -f "$json_file" ]; then
json_name=$(basename "$json_file" .json)

# Skip the SDK's own nativesdk triple. It shares the architecture prefix
# with the device triple whenever the SDK host arch matches the target
# arch, which on x86_64 means RUST_TARGET_PATH holds both
# x86_64-avocado-linux-gnu and x86_64-avocadosdk-linux-gnu. The
# avocadosdk vendor is host-side by construction - avocado-cli tags every
# nativesdk artifact <arch>_avocadosdk - so it is never a candidate for a
# device binary and must not count as a competing match.
case "$json_name" in
"${OECORE_TARGET_ARCH}-avocadosdk-"*) continue ;;
esac

if [[ "$json_name" == "${OECORE_TARGET_ARCH}-"* ]]; then
RUST_TARGET="$json_name"
match_count=$((match_count + 1))
matches="$matches $json_name"
fi
fi
done
# This fetches a release artifact rather than compiling from source, which is the
# same shape ext-microclaw uses. The reason is not laziness about cross-compiling:
# `avocado ext package` runs no build step, so the published RPM can only carry
# what is in the repo. Shipping the crate source instead pushed the compile onto
# the consumer, and a consumer has no reason to have a Rust cross toolchain in
# their SDK. Measured 2026-09-15: a clean project following the documented setup
# failed inside ring's build script with
# `ToolNotFound: failed to find tool "x86_64-avocado-linux-gcc"`, after both
# `avocado install` and the extension's own sysroot creation had reported success.
#
# The binaries are statically linked against musl, so they carry no libc version
# coupling and run on any Avocado target of the right architecture regardless of
# what the device's glibc is.
set -euo pipefail

if [ -z "$RUST_TARGET" ]; then
echo "Error: Could not find Rust target for $OECORE_TARGET_ARCH" >&2
VERSION="${AVOCADO_EXT_VERSION:-}"
if [ -z "$VERSION" ]; then
VERSION=$(grep -oE '^[[:space:]]+version:[[:space:]]*[0-9]+\.[0-9]+\.[0-9]+' \
"$(dirname "$0")/avocado.yaml" | head -1 | grep -oE '[0-9]+\.[0-9]+\.[0-9]+')
fi
if [ -z "$VERSION" ]; then
echo "Error: could not determine the extension version" >&2
exit 1
fi

if [ "$match_count" -gt 1 ]; then
echo "Error: $OECORE_TARGET_ARCH matches $match_count Rust targets:$matches" >&2
echo "Error: refusing to pick one - the wrong triple yields a binary that packages and installs but cannot exec on the device." >&2
# Map the SDK's architecture onto the published artifact. An unmapped
# architecture fails loudly here rather than installing nothing and leaving the
# agent silently absent from the image.
case "${OECORE_TARGET_ARCH:-}" in
x86_64) ASSET_ARCH="x86_64-unknown-linux-musl" ;;
aarch64) ASSET_ARCH="aarch64-unknown-linux-musl" ;;
"")
echo "Error: OECORE_TARGET_ARCH is unset; cannot choose an artifact" >&2
exit 1
fi
;;
*)
echo "Error: no prebuilt agent for OECORE_TARGET_ARCH=${OECORE_TARGET_ARCH}" >&2
echo "Published artifacts cover x86_64 and aarch64 only. Add the architecture" >&2
echo "to the binaries matrix in .github/workflows/release.yml and re-release." >&2
exit 1
;;
esac

# Stable marker: the resolution fixture test greps for this to tell "resolution
# succeeded and a later step failed" from "resolution itself failed", which exit
# code alone cannot distinguish when the test runs without a real SDK. Load-
# bearing test surface, not a debug echo.
echo "resolved-target: $RUST_TARGET"
ASSET="avocado-container-agent-dev-${VERSION}-${ASSET_ARCH}.tar.gz"
BASE="https://github.com/avocado-linux/ext-container-agent-dev/releases/download/${VERSION}"

echo "Compiling avocado-container-agent-dev for target: $RUST_TARGET"
STAGE_DIR="${AVOCADO_BUILD_DIR}/agent"
mkdir -p "$STAGE_DIR"
cd "$STAGE_DIR"

cd "$(dirname "$0")/agent"
echo "resolved-artifact: ${ASSET}"

if [ ! -f "$ASSET" ]; then
echo "Fetching ${BASE}/${ASSET}"
curl -fsSL -o "$ASSET" "${BASE}/${ASSET}"
fi

# Clear any rustflags that might cause conflicts with our .cargo/config.toml
unset RUSTFLAGS
unset CARGO_BUILD_RUSTFLAGS
for var in $(env | grep -o 'CARGO_TARGET_[A-Z0-9_]*_RUSTFLAGS'); do
unset "$var"
done
# SHA256SUMS comes from the same release as the artifact, so this verifies the
# transfer rather than the provenance - the release itself is the trust anchor.
# That is proportionate for a first-party artifact and avoids pinning a digest
# in this file, which would force every release through a build-commit-tag dance
# just to record the hash of something the tag itself produces.
echo "Fetching ${BASE}/SHA256SUMS"
curl -fsSL -o SHA256SUMS "${BASE}/SHA256SUMS"

# Remove any existing config that might conflict
rm -rf .cargo
if ! grep -q " ${ASSET}\$" SHA256SUMS; then
echo "Error: SHA256SUMS from the ${VERSION} release does not list ${ASSET}" >&2
echo "The release is incomplete for this architecture; do not install it." >&2
exit 1
fi

# Create config.toml with cross-compilation settings
mkdir -p .cargo
cat > .cargo/config.toml << EOF
[target.$RUST_TARGET]
rustflags = ["--sysroot=$SDKTARGETSYSROOT/usr", "-C", "link-arg=--sysroot=$SDKTARGETSYSROOT"]
EOF
grep " ${ASSET}\$" SHA256SUMS | sha256sum -c -

# Use a persistent cargo registry cache to avoid re-downloading crates
export CARGO_HOME="${AVOCADO_BUILD_DIR}/.cargo-cache"
tar -xzf "$ASSET"

# Enforce the ring-only rustls crypto provider before building. TARGET is
# mandatory here so the guard resolves the same SDK triple the build ships;
# without it the guard would check the host triple instead. Runs under set -e,
# so a non-zero exit (aws-lc-rs present, or a guard error) fails the compile.
TARGET="$RUST_TARGET" scripts/check-provider.sh
cargo build --release --target "$RUST_TARGET" --target-dir "$AVOCADO_BUILD_DIR"
if [ ! -f avocado-container-agent-dev ]; then
echo "Error: ${ASSET} did not contain avocado-container-agent-dev" >&2
exit 1
fi

echo "avocado-container-agent-dev compiled successfully"
chmod 0755 avocado-container-agent-dev
echo "avocado-container-agent-dev ${VERSION} staged for ${ASSET_ARCH}"
Loading
Loading