Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ jobs:
# it cross-builds that target from Linux too.

# aarch64-apple-darwin cross-compiles from the Linux runner via
# soldr + cargo-zigbuild + soldr's managed Apple SDK, matching the
# soldr's blessed cross path + its managed Apple SDK, matching the
# release matrix. No macos-latest runner is used anywhere in this
# workflow.
- target: aarch64-apple-darwin
Expand Down
8 changes: 8 additions & 0 deletions .github/workflows/ci-workflow-drift.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,3 +42,11 @@ jobs:
run: uv run --with pyyaml --no-project python ci/check_workflow_concurrency.py
- name: Unit-test the concurrency guard
run: uv run --with pyyaml --no-project python -m unittest ci.test_workflow_concurrency
# soldr's blessed cross path is the only sanctioned one; cargo-zigbuild,
# ziglang, zig cc and cargo-xwin are retired. This gate keeps them from
# coming back via a copy-pasted snippet or a half-remembered recipe.
# See agents/docs/cross-compilation.md.
- name: Verify the retired cross backends stay purged
run: uv run --no-project python ci/check_no_legacy_cross.py
- name: Unit-test the cross-backend guard
run: uv run --no-project python -m unittest ci.test_no_legacy_cross
4 changes: 2 additions & 2 deletions .github/workflows/release-auto.yml
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ jobs:
# every fbuild workflow pins the binary version explicitly.
#
# aarch64-apple-darwin cross-compiles from the Linux
# runner via soldr + cargo-zigbuild + soldr's managed
# runner via soldr's blessed cross path + its managed
# Apple SDK — no mac-host dependency in the matrix.
- target: aarch64-apple-darwin
runner: ubuntu-latest
Expand All @@ -197,7 +197,7 @@ jobs:
# x86_64-apple-darwin shares the mac_cross_linux template
# branch — soldr's apple_sdk fetcher auto-picks
# `darwin-x86_64` (thin-x86_64) when the target triple is
# `x86_64-apple-darwin`, and `cargo zigbuild --target
# `x86_64-apple-darwin`, and `soldr build --target
# x86_64-apple-darwin` resolves the SDKROOT + frameworks
# identically to the arm64 lane. Closes Lane 3 of soldr#1006.
- target: x86_64-apple-darwin
Expand Down
276 changes: 63 additions & 213 deletions .github/workflows/template_native_build.yml

Large diffs are not rendered by default.

12 changes: 9 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ When operating in this repo on a task that isn't covered by the architectural ov
| "What DTR/RTS state do I open this CDC port at?" | [`docs/usb-cdc-control-line-matrix.md`](docs/usb-cdc-control-line-matrix.md) |
| "How do I run the serial detection code against a real ESP32?" | [`agents/docs/serial-testing.md`](agents/docs/serial-testing.md) (FastLED/fbuild#899 — Docker/WSL real-device harness) |
| "Where does this path/cache/build dir live, and why won't my cache key hit?" | [`agents/docs/path-conventions.md`](agents/docs/path-conventions.md) |
| "How do I cross-compile / why did a release lane break?" | [`agents/docs/cross-compilation.md`](agents/docs/cross-compilation.md) |
| "Which crate owns this code?" | [`crates/CLAUDE.md`](crates/CLAUDE.md) |
| "Which architecture doc maps to my crate?" | [`docs/CLAUDE.md`](docs/CLAUDE.md) |
| "Is this serial port the right device?" | `fbuild serial probe list` (FastLED/fbuild#686) |
Expand Down Expand Up @@ -61,9 +62,14 @@ The four rules an agent must internalize before doing anything else (all listed
- MSRV: 1.95.0 | Edition: 2021 | Toolchain: 1.95.0 pinned in `rust-toolchain.toml` (clippy + rustfmt)
- CI hosts: Linux, Windows. All warnings denied (`RUSTFLAGS="-D warnings"`). There are
**no macOS runners** — macOS is a build *target*, not a test host: every apple-darwin
binary is cross-built from Linux (soldr + `cargo-zigbuild` + managed Apple SDK) by the
Linux-hosted native workflows, `release-auto.yml` and `build.yml`. Do not add a
`macos-latest` lane back.
binary is cross-built from Linux by the Linux-hosted native workflows,
`release-auto.yml` and `build.yml`. Do not add a `macos-latest` lane back.
- **Cross-compilation goes through soldr, and only soldr** — `soldr prepare --target X`
then `soldr build --target X`. The zig- and xwin-based wrappers are retired and
enforced-out by `ci/check_no_legacy_cross.py` plus the `tool_guard.py` hook. Never pass zigbuild's `.2.17` glibc suffix to soldr: it has no
such target, warns, falls back to the host toolchain and still exits 0, shipping a
GLIBC_2.39 wheel tagged `manylinux_2_17`. soldr's own sysroot links at 2.16.
See [`agents/docs/cross-compilation.md`](agents/docs/cross-compilation.md).
- Every directory with files must have a README.md (enforced by hook)

## Commands
Expand Down
16 changes: 16 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,22 @@ exclude = [
[workspace.metadata.dylint]
libraries = [{ path = "dylints/*" }]

# Release targets, in the shape `soldr prepare --target all` expands. soldr's
# blessed cross path builds every one of these from a Linux runner; the repo
# no longer carries cargo-zigbuild or cargo-xwin for any of them.
# See agents/docs/cross-compilation.md.
[workspace.metadata.soldr]
targets = [
"x86_64-unknown-linux-musl",
"aarch64-unknown-linux-musl",
"x86_64-unknown-linux-gnu",
"aarch64-unknown-linux-gnu",
"x86_64-apple-darwin",
"aarch64-apple-darwin",
"x86_64-pc-windows-msvc",
"aarch64-pc-windows-msvc",
]

[workspace.package]
version = "2.5.22"
edition = "2021"
Expand Down
5 changes: 5 additions & 0 deletions agents/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,11 @@ routing table sends an agent here by task.
factory functions that pick them, and why a mis-spelled/absolute path
silently defeats a cache key. **Read before touching any cache dir,
build dir, or cache-key/signature code** (FastLED/fbuild#952).
- [`cross-compilation.md`](cross-compilation.md) — soldr's blessed
cross path (`soldr prepare` + `soldr build`), the banned wrappers and
the lint that enforces it, and the **manylinux glibc-floor trap**: a
wheel can build green, be tagged `manylinux_2_17`, and still require
glibc 2.39. **Read before touching any release lane.**
- [`deploy-architecture.md`](deploy-architecture.md) — the
`Deployer` trait, `post_deploy_recovery`, board-family dispatch
model.
Expand Down
138 changes: 138 additions & 0 deletions agents/docs/cross-compilation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
# Cross-compilation

How fbuild's release binaries and wheels get built for platforms the
runner is not. **Read this before touching
[`.github/workflows/template_native_build.yml`](../../.github/workflows/template_native_build.yml)
or any release lane.**

## The blessed path

soldr owns cross-compilation. Two commands, identical shape for every
supported triple:

```bash
soldr prepare --target <triple> # stdlib + compiler/linker + SDK/sysroot + env
soldr build --release --target <triple> -p <crate>
```

`soldr prepare --help` states the contract: it "installs the Rust
standard library, selects and materializes the blessed compiler/linker
plus SDK or sysroot, and exports the target-scoped environment. Legacy
backend wrappers are diagnostic-only overrides and are never selected by
this command."

**`cargo-zigbuild`, `ziglang`, `zig cc`/`zig c++`, and `cargo-xwin` are
banned**, enforced by `ci/check_no_legacy_cross.py` (unit tests in
`ci/test_no_legacy_cross.py`). The gate scans runnable lines only —
comments may name them, which is how the history below stays readable.

soldr's supported targets, as reported by its own error text when asked
for something else:

```
x86_64-pc-windows-msvc, x86_64-pc-windows-gnu, aarch64-pc-windows-msvc,
x86_64-apple-darwin, aarch64-apple-darwin,
x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu,
x86_64-unknown-linux-musl, aarch64-unknown-linux-musl
```

## The trap: never hand soldr a zigbuild target suffix

**This is the mistake not to repeat.** It cost hours and nearly cost a
lane of the migration.

manylinux wheels are glibc-based: a `manylinux_2_17` wheel promises it
runs on glibc >= 2.17. Nothing in the build or the packaging checks that
promise -- `ci/publish.py` assigns the tag from a filename map, not by
measuring the binary. So a `.so` linked against a newer glibc gets tagged
`manylinux_2_17`, uploads fine, installs fine, and fails at **import**
on any distro older than the build host.

**soldr holds the floor by itself.** Its catalogue sysroot for
`x86_64-unknown-linux-gnu` produces an extension topping out at
**GLIBC_2.16** -- below the 2.17 floor, and below what the retired
`cargo zigbuild --target ...-gnu.2.17` lane produced:

| Built with | Max GLIBC symbol |
|---|---|
| `cargo zigbuild --target x86_64-unknown-linux-gnu.2.17` (retired) | 2.17 |
| `soldr build --target x86_64-unknown-linux-gnu` | **2.16** |
| `soldr build --target x86_64-unknown-linux-gnu.2.17` | **2.39** |

Read that third row again. **Do not port zigbuild's `.<major>.<minor>`
suffix onto a soldr target.** soldr has no such target. It logs the miss
as a *warning*, falls back to the bare host toolchain, and still exits 0:

```
soldr build: catalogue zstd sysroot unavailable for x86_64-unknown-linux-gnu.2.17:
unsupported platform: no zstd sysroot recipe for target x86_64-unknown-linux-gnu.2.17;
supported: [... "x86_64-unknown-linux-gnu" ...]
error: error loading target specification:
could not find specification for target "x86_64-unknown-linux-gnu.2.17"
... exit code 0
```

The output lands in `target/.../x86_64-unknown-linux-gnu/release/` with
the suffix normalized away, so the path looks right too. The only signal
that anything went wrong is the glibc floor of the artifact.

Carrying a habit from the old backend into the new one produced a
*worse* result than either doing nothing or doing it right. When
migrating a toolchain, re-derive the invocation from the new tool's own
docs; do not translate the old flags.

## Always verify the artifact, never the exit code

Both failure modes in this doc produced **exit 0**. Check the binary:

```bash
# glibc floor of a Linux .so — anything above 2.17 is a broken wheel
objdump -T <lib>.so | grep -o 'GLIBC_[0-9.]*' | sort -Vu | tail -4

# Mach-O arch (no `file` on some hosts): cffaedfe = Mach-O 64 LE,
# cputype 0x100000c = arm64, 0x1000007 = x86_64
python3 -c "d=open('<bin>','rb').read(8); print(d[:4].hex(), hex(int.from_bytes(d[4:8],'little')))"
```

The published wheel is downloadable, so the shipped floor is always
checkable after the fact:

```bash
curl -sL <wheel-url> -o w.whl && python3 -c "import zipfile;zipfile.ZipFile('w.whl').extractall('x')"
objdump -T x/fbuild/_native.abi3.so | grep -o 'GLIBC_[0-9.]*' | sort -Vu | tail
```

## Pin every toolchain that can float

On 2026-09-03 the 2.5.22 release failed both apple-darwin lanes:

```
error: unable to read exported symbols list '-dead_strip': FileNotFound
error: could not compile `zccache-watcher` (lib) due to 1 previous error
```

Cause: the template ran `pip install cargo-zigbuild` **unpinned**. The
last good release built on 0.23.1; that day pip served 0.23.4, which
reorders the `-Wl,-exported_symbols_list` / `-Wl,<path>` pair rustc
emits for a cdylib so zig reads the following flag as the list path.
Nothing in fbuild had changed.

The rule: **a release lane may not install a floating version of
anything.** soldr's binary version is pinned explicitly in the workflow;
any remaining pip install is pinned too. When a pin moves, prove it with
a release build before merging.

## Why the version pin matters more than it looks

`zackees/setup-soldr@v0` is a floating major tag; the `version:` input
pins only the soldr *binary*, not the action. If a cross lane breaks
with no corresponding fbuild change, compare the action SHA and the
installed tool versions against the last good run before touching
fbuild's own code:

```bash
gh run view --job <id> --repo FastLED/fbuild --log | grep -E "Download action repository|Successfully installed"
```

That diff is what identified the cargo-zigbuild regression above in
minutes.
Loading
Loading