From 0fc88ba1a23e072b07b98716c5f3d5f4c9cb05f8 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 2 Sep 2026 11:38:34 +0800 Subject: [PATCH 01/12] platform: carry the spawn errno up instead of re-spawning; run_exec is never silent (#544) --- ...-02-issue544-runner-implementation-plan.md | 1142 +++++++++++++++++ ...26-09-02-runner-beyond-baremetal-design.md | 721 +++++++++++ modules/platform/src/process.cppm | 114 +- .../platform/src/unix/bounded_process.cppm | 10 +- .../platform/src/windows/bounded_process.cppm | 8 +- tests/unit/test_process_run_exec.cpp | 108 ++ 6 files changed, 2079 insertions(+), 24 deletions(-) create mode 100644 .agents/docs/2026-09-02-issue544-runner-implementation-plan.md create mode 100644 .agents/docs/2026-09-02-runner-beyond-baremetal-design.md diff --git a/.agents/docs/2026-09-02-issue544-runner-implementation-plan.md b/.agents/docs/2026-09-02-issue544-runner-implementation-plan.md new file mode 100644 index 00000000..9e255b75 --- /dev/null +++ b/.agents/docs/2026-09-02-issue544-runner-implementation-plan.md @@ -0,0 +1,1142 @@ +# Issue #544: runner beyond bare metal — implementation plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** `[target.].runner` is honoured on every target, a spawn failure is never silent, `mcpp test` reports unrunnable artifacts as not-run with exit 2, `[xlings]` values can be given per host platform, and the whole change ships as one PR, one release, and one sandbox verification. + +**Architecture:** `choose_runner` stays the single read point and stops gating on `os == none`; the launcher layer carries the spawn errno up instead of re-spawning; the manifest resolves `[xlings]` per-platform values against the host at load time so every downstream reader keeps seeing a flat list. Design: `.agents/docs/2026-09-02-runner-beyond-baremetal-design.md` (sections 0 and 4 are normative). + +**Tech Stack:** C++23 modules (mcpp builds itself with `mcpp build`), gtest unit tests under `tests/unit/` run by `mcpp test`, bash e2e under `tests/e2e/` run by `tests/e2e/run_all.sh` or individually with `MCPP= bash tests/e2e/.sh`. + +## Global Constraints + +- Baseline is `origin/main` at `aeba151` (version `2026.9.1.1`). Work in a fresh worktree; never switch branches in the shared checkout. +- All code goes through one PR with squash merge; CI must be green on the PR and on `origin/main` after merge. +- Prose in docs and the PR is English academic register, declarative, no emoji, no question headings (`.agents/skills/mcpp-docs-style/SKILL.md`). Every `docs/*.md` change has a `docs/zh/` twin; CI enforces the pair (`.github/tools/check_docs_style.sh`). +- New e2e scripts carry no `# requires:` guard beyond `unix-shell`; both CI shards lack llvm and a `requires` skip exits 0. +- Exit code 2 means "could not start"; 1 keeps meaning "ran and failed"; 0 means every test ran and passed. +- Platform keys accepted in `[xlings]` values: `linux`, `macos` (alias `macosx`), `windows`, `default`. Any other key is a hard manifest error. +- Version bump: `mcpp.toml` `[package].version` and `modules/versioning/src/version.cppm` `MCPP_VERSION` in the same commit; `.xlings.json` bootstrap pin stays at the released version until the new release is indexed. + +--- + +### Task 1: Worktree and baseline + +**Files:** none modified. + +- [ ] **Step 1: Create the worktree from `origin/main`** + +```bash +cd ~/workspace/github/mcpp-community/mcpp +git fetch origin +git worktree add ../mcpp-issue544 -b fix/runner-hosted-targets origin/main +cd ../mcpp-issue544 +git log --oneline -1 # must be aeba151 +``` + +- [ ] **Step 2: Build the engine and run the unit suite once** + +```bash +mcpp self config --mirror CN +mcpp build 2>&1 | tail -3 +mcpp test unit/test_process_run_exec 2>&1 | tail -3 +mcpp test unit/test_manifest 2>&1 | tail -3 +``` + +Expected: both `ok.` The built binary is `target///bin/mcpp`; record its path as `$MCPP` for the e2e runs below (`MCPP=$(ls -d target/*/*/bin/mcpp | head -1)`). + +--- + +### Task 2: Launcher carries the spawn errno; no second spawn; `run_exec` never silent + +**Files:** +- Modify: `modules/platform/src/unix/bounded_process.cppm:59-66` (struct), `:227-230` (spawn site) +- Modify: `modules/platform/src/windows/bounded_process.cppm:67-77` (struct), the `CreateProcess` failure site +- Modify: `modules/platform/src/process.cppm` — `run_exec` (:552-577), `capture_exec` (:580-625), `BoundedOutcome` (:661-666), `dispatch_bounded` (:707-726), `run_exec_deadline` (:731-750), `capture_exec_deadline` (:840-862), and the exported declarations near `:110` and `:175` +- Test: `tests/unit/test_process_run_exec.cpp` + +**Interfaces:** +- Produces: every launcher gains a trailing `int* spawn_error = nullptr`. Contract: when the child could not be spawned, the launcher returns 127 (or `exit_code = 127` in `RunResult`) and (a) if `spawn_error` is non-null, stores the errno there and prints nothing; (b) if null, prints `spawn_failure(argv.front(), err)` to stderr (`run_exec`) or into `output` (`capture_exec`). `DeadlineRun` and `BoundedOutcome` gain `int spawn_error = 0`; `supported == false && spawn_error != 0` means "spawned attempted and refused" and the deadline wrappers do not fall back in that case. + +- [ ] **Step 1: Write the failing tests** + +Append to `tests/unit/test_process_run_exec.cpp` inside the `#if !defined(_WIN32)` block: + +```cpp +TEST(RunExec, MissingProgramReportsOnStderrWhenCallerDoesNotAsk) { + testing::internal::CaptureStderr(); + int rc = process::run_exec({"/no/such/program/mcpp-run-xyz"}); + auto err = testing::internal::GetCapturedStderr(); + EXPECT_EQ(rc, 127); + EXPECT_NE(err.find("/no/such/program/mcpp-run-xyz"), std::string::npos) << err; + EXPECT_NE(err.find("error 2"), std::string::npos) << err; +} + +TEST(RunExec, MissingProgramTypesErrnoWhenCallerAsks) { + int spawnErr = 0; + testing::internal::CaptureStderr(); + int rc = process::run_exec({"/no/such/program/mcpp-run-xyz"}, {}, &spawnErr); + auto err = testing::internal::GetCapturedStderr(); + EXPECT_EQ(rc, 127); + EXPECT_EQ(spawnErr, ENOENT); + EXPECT_TRUE(err.empty()) << err; // the caller owns the report +} + +TEST(RunExec, UnloadableArtifactIsENOEXEC) { + // A file with the executable bit and no loader magic: the kernel refuses + // it with ENOEXEC. posix_spawnp does not retry through /bin/sh the way + // execvp does, so the errno reaches the caller untouched. + auto dir = std::filesystem::temp_directory_path() / "mcpp-enoexec-test"; + std::filesystem::create_directories(dir); + auto f = dir / "not-an-elf"; + { std::ofstream o(f); o << "\x7f" "NOT"; } + std::filesystem::permissions(f, std::filesystem::perms::owner_all); + int spawnErr = 0; + int rc = process::run_exec({f.string()}, {}, &spawnErr); + EXPECT_EQ(rc, 127); + EXPECT_EQ(spawnErr, ENOEXEC); +} + +TEST(RunExecDeadline, SpawnFailureIsTypedAndNotRetried) { + int spawnErr = 0; + bool timedOut = false; + testing::internal::CaptureStderr(); + int rc = process::run_exec_deadline({"/no/such/program/mcpp-dl-xyz"}, {}, + std::chrono::milliseconds(5000), &timedOut, + &spawnErr); + auto err = testing::internal::GetCapturedStderr(); + EXPECT_EQ(rc, 127); + EXPECT_EQ(spawnErr, ENOENT); + EXPECT_FALSE(timedOut); + EXPECT_TRUE(err.empty()) << err; +} + +TEST(CaptureExecDeadline, SpawnFailureIsTypedInOutcome) { + int spawnErr = 0; + bool timedOut = false; + auto r = process::capture_exec_deadline({"/no/such/program/mcpp-cdl-xyz"}, {}, + std::chrono::milliseconds(5000), &timedOut, + {}, &spawnErr); + EXPECT_EQ(r.exit_code, 127); + EXPECT_EQ(spawnErr, ENOENT); + EXPECT_TRUE(r.output.empty()) << r.output; +} +``` + +Add `#include ` and `#include ` at the top of the file. + +- [ ] **Step 2: Run to verify they fail** + +Run: `mcpp build && mcpp test unit/test_process_run_exec` +Expected: compile failure on the new `spawn_error` parameter (the tests name a signature that does not exist yet). + +- [ ] **Step 3: Implement** + +`modules/platform/src/unix/bounded_process.cppm`, struct: + +```cpp +struct DeadlineRun { + bool supported = false; + int exit_code = 0; + bool timed_out = false; + // Non-zero when `supported` is false BECAUSE the spawn was attempted and + // refused (the errno posix_spawnp returned). Zero with `supported` false + // means the platform has no bounded launcher at all. The two need + // opposite treatment upstream: the first is reported, the second falls + // back to the unbounded launcher. + int spawn_error = 0; +}; +``` + +Spawn site (`:230`): `if (sp != 0) { out.spawn_error = sp; if (capture) ::close(fds[0]); return out; }`. + +`modules/platform/src/windows/bounded_process.cppm`: same field; at the `CreateProcessW` failure site set `out.spawn_error = static_cast(::GetLastError());`. + +`modules/platform/src/process.cppm`: + +```cpp +struct BoundedOutcome { + bool supported = false; + int exit_code = 0; + bool timed_out = false; + int spawn_error = 0; + std::string output; +}; +// dispatch_bounded: copy r.spawn_error into outcome.spawn_error on both branches. + +int run_exec(const std::vector& argv, + const std::vector>& extraEnv, + int* spawn_error) +{ + if (spawn_error) *spawn_error = 0; + if (argv.empty()) return 127; +#if defined(__linux__) || defined(__APPLE__) + ... + pid_t pid = 0; + if (int sp = ::posix_spawnp(&pid, cargv[0], nullptr, nullptr, cargv.data(), envp.data()); + sp != 0) { + // A spawn failure is reported exactly once: by the caller when it + // asked to see the errno, and here otherwise. It is never dropped — + // "Running …", a blank line and exit 1 was the whole of #544's output. + if (spawn_error) *spawn_error = sp; + else std::fputs(spawn_failure(argv.front(), sp).c_str(), stderr); + return 127; + } + ... +#else + // std::system cannot distinguish a refused launch from a failed child; + // spawn_error stays 0 and the shell's own message is what the user sees. + ... +#endif +} +``` + +`capture_exec`: same shape; when `spawn_error` is non-null store and leave `output` empty, else format `spawn_failure` into `output` as today. + +`run_exec_deadline` and `capture_exec_deadline`: + +```cpp + auto r = dispatch_bounded(...); + if (!r.supported) { + if (r.spawn_error != 0) { + // Attempted and refused. Spawning again would only discard this + // errno and pay for a second refusal. + if (spawn_error) *spawn_error = r.spawn_error; + else std::fputs(spawn_failure(argv.front(), r.spawn_error).c_str(), stderr); + return 127; // capture variant: result.exit_code = 127 + } + return run_exec(argv, extraEnv, spawn_error); // no bounded launcher here + } +``` + +Update the exported declarations (`process.cppm:110`, `:175`, and the `run_exec`/`capture_exec` declarations) with the trailing `int* spawn_error = nullptr`. `run_shell_deadline` is unchanged. + +- [ ] **Step 4: Run to verify they pass** + +Run: `mcpp build && mcpp test unit/test_process_run_exec` +Expected: `ok.` with the five new tests listed. + +- [ ] **Step 5: Commit** + +```bash +git add modules/platform tests/unit/test_process_run_exec.cpp +git commit -m "platform: carry the spawn errno up instead of re-spawning; run_exec is never silent (#544)" +``` + +--- + +### Task 3: Manifest — array keys in the sweep; `[xlings]` per-platform values + +**Files:** +- Modify: `modules/manifest/src/toml.cppm:1350-1361` (`[xlings]`), `:1917-1959` (sweep) +- Modify: `modules/manifest/src/types.cppm:758-775` (`XlingsConfig`, comment only) +- Test: `tests/unit/test_manifest.cpp` + +**Interfaces:** +- Produces: `mcpp::manifest::resolve_host_value(const toml::Value&, std::string_view hostPlatform) -> std::expected, std::string>` exported from the manifest module. Returns the string, `nullopt` when the value is a platform table with no matching key and no `default`, and an error string for a table with an unknown key or a non-string leaf. `mcpp::manifest::host_platform_key()` returns `"linux"`, `"macos"` or `"windows"` for the running binary. `XlingsConfig::deps` and `::workspace` keep their flat types; resolution happens at load. + +- [ ] **Step 1: Write the failing tests** + +Append to `tests/unit/test_manifest.cpp` (follow the file's fixture helper for writing a manifest to a temp dir and calling `mcpp::manifest::load`; the helper used at `:3249` is the model): + +```cpp +TEST(Manifest, TargetSweepReportsArrayTyposAndListsRunner) { + constexpr auto src = R"( +[package] +name = "app" +version = "0.1.0" +[target.aarch64-linux-musl] +runnerX = ["qemu-aarch64-static"] +)"; + auto m = load_from_string(src); + ASSERT_TRUE(m.has_value()) << m.error().message; + ASSERT_EQ(m->schemaWarnings.size(), 1u); + EXPECT_NE(m->schemaWarnings[0].find("'runnerX'"), std::string::npos); + EXPECT_NE(m->schemaWarnings[0].find("runner"), std::string::npos); + EXPECT_NE(m->schemaWarnings[0].find("Supported keys: cxx_runtime, linkage, runner, sysroot, toolchain"), + std::string::npos) << m->schemaWarnings[0]; +} + +TEST(Manifest, XlingsDepsAcceptPerPlatformEntries) { + constexpr auto src = R"( +[package] +name = "app" +version = "0.1.0" +[xlings] +deps = ["xim:ninja", { linux = "qemu-user-aarch64" }, { windows = "nasm", default = "yasm" }] +)"; + auto m = load_from_string(src); + ASSERT_TRUE(m.has_value()) << m.error().message; + const auto host = mcpp::manifest::host_platform_key(); + std::vector want{"xim:ninja"}; + if (host == "linux") want.push_back("qemu-user-aarch64"); + want.push_back(host == "windows" ? "nasm" : "yasm"); + EXPECT_EQ(m->xlings.deps, want); +} + +TEST(Manifest, XlingsWorkspaceAcceptsPerPlatformValues) { + constexpr auto src = R"( +[package] +name = "app" +version = "0.1.0" +[xlings.workspace] +gcc = { linux = "15.1.0" } +llvm = { macos = "20", default = "22" } +)"; + auto m = load_from_string(src); + ASSERT_TRUE(m.has_value()) << m.error().message; + const auto host = mcpp::manifest::host_platform_key(); + if (host == "linux") EXPECT_EQ(m->xlings.workspace.at("gcc"), "15.1.0"); + else EXPECT_EQ(m->xlings.workspace.count("gcc"), 0u); + EXPECT_EQ(m->xlings.workspace.at("llvm"), host == "macos" ? "20" : "22"); +} + +TEST(Manifest, XlingsUnknownPlatformKeyIsAnError) { + constexpr auto src = R"( +[package] +name = "app" +version = "0.1.0" +[xlings] +deps = [{ linxu = "qemu-user-aarch64" }] +)"; + auto m = load_from_string(src); + ASSERT_FALSE(m.has_value()); + EXPECT_NE(m.error().message.find("linxu"), std::string::npos) << m.error().message; + EXPECT_NE(m.error().message.find("linux, macos, windows, default"), std::string::npos) + << m.error().message; +} + +TEST(Manifest, ResolveHostValueTable) { + using mcpp::manifest::resolve_host_value; + auto d = mcpp::libs::toml::parse(R"(v = { linux = "a", macosx = "b", default = "c" })"); + ASSERT_TRUE(d.has_value()); + const auto& v = d->root().at("v"); + EXPECT_EQ(resolve_host_value(v, "linux").value().value(), "a"); + EXPECT_EQ(resolve_host_value(v, "macos").value().value(), "b"); // macosx alias + EXPECT_EQ(resolve_host_value(v, "windows").value().value(), "c"); + auto e = mcpp::libs::toml::parse(R"(v = { linux = "a" })"); + EXPECT_FALSE(resolve_host_value(e->root().at("v"), "windows").value().has_value()); +} +``` + +If `test_manifest.cpp` has no `load_from_string` helper, add one at the top of the file that writes `src` to `/mcpp.toml` and calls `mcpp::manifest::load(path)`; model it on the fixture at `:3240-3250`. + +- [ ] **Step 2: Run to verify they fail** + +Run: `mcpp build && mcpp test unit/test_manifest` +Expected: compile failure (`host_platform_key`, `resolve_host_value` undefined). + +- [ ] **Step 3: Implement** + +In `modules/manifest/src/toml.cppm`, before `load`: + +```cpp +// The host platform in the vocabulary xlings' `.xlings.json` uses for a +// per-platform value, spelled with mcpp's OS names. `[xlings]` describes the +// environment of the machine mcpp runs on, so it is resolved against that +// machine when the manifest is loaded; every downstream reader keeps seeing a +// flat list, which is what keeps the provisioning pass, its stamp and the +// build-program hand-off unchanged. +export std::string_view host_platform_key() { +#if defined(_WIN32) + return "windows"; +#elif defined(__APPLE__) + return "macos"; +#else + return "linux"; +#endif +} + +// A value that is either a string or a table keyed by platform, as xlings' +// `workspace` allows: `"20"` or `{ linux = "15.1.0", default = "22" }`. +// `macosx` is accepted as xlings' own spelling of `macos`. A table with no +// entry for this host and no `default` resolves to nullopt, which the caller +// treats as "not declared on this host" — the same rule xlings applies. +// An unknown key is an error, not a dropped entry: a typo'd platform that +// silently declared nothing is the shape #531 was filed for. +export std::expected, std::string> +resolve_host_value(const mcpp::libs::toml::Value& v, std::string_view host) { + if (v.is_string()) return std::optional{v.as_string()}; + if (!v.is_table()) + return std::unexpected("expected a string or a { = \"...\" } table"); + static constexpr std::string_view kKnown[] = {"linux", "macos", "macosx", "windows", "default"}; + std::optional chosen, fallback; + for (auto& [k, val] : v.as_table()) { + bool known = false; + for (auto n : kKnown) if (k == n) { known = true; break; } + if (!known) + return std::unexpected(std::format( + "unknown platform key '{}'; expected one of linux, macos, windows, default", k)); + if (!val.is_string()) + return std::unexpected(std::format("platform key '{}' must be a string", k)); + std::string_view canon = (k == "macosx") ? "macos" : std::string_view(k); + if (canon == host) chosen = val.as_string(); + else if (canon == "default") fallback = val.as_string(); + } + if (chosen) return chosen; + return fallback; // may be nullopt: not declared on this host +} +``` + +Replace the `[xlings]` block at `:1350-1361`: + +```cpp + // [xlings] — build environment (L-1). Subsections mirror .xlings.json 1:1, + // including its per-platform value form, resolved here for this host. + if (auto* arr = doc->get("xlings.deps"); arr && arr->is_array()) { + std::size_t i = 0; + for (auto& el : arr->as_array()) { + auto r = resolve_host_value(el, host_platform_key()); + if (!r) return std::unexpected(error(origin, + std::format("[xlings] deps[{}]: {}", i, r.error()))); + if (*r) m.xlings.deps.push_back(**r); + ++i; + } + } + ... subos unchanged ... + if (auto* wt = doc->get_table("xlings.workspace")) + for (auto& [k, val] : *wt) { + auto r = resolve_host_value(val, host_platform_key()); + if (!r) return std::unexpected(error(origin, + std::format("[xlings.workspace] {}: {}", k, r.error()))); + if (*r) m.xlings.workspace[k] = **r; + } +``` + +(`doc->get` returns the raw value; check the `mcpp.libs.toml` accessor names in `modules/libs/toml` before writing — `get`, `get_table`, `as_array`, `is_table` are the names used elsewhere in this file.) + +Sweep at `:1936-1959`: + +```cpp + static constexpr std::string_view kKnownTargetScalars[] = { + "cxx_runtime", "linkage", "sysroot", "toolchain", + }; + static constexpr std::string_view kKnownTargetArrays[] = { "runner" }; + for (auto& [key, value] : body) { + if (value.is_table()) continue; // the conditional channel + const auto& known = value.is_array() + ? std::span(kKnownTargetArrays) + : std::span(kKnownTargetScalars); + if (std::ranges::find(known, key) != known.end()) continue; + // The list names every key this table reads, arrays included, + // in one alphabetical line: a reader of the warning should not + // have to know which type a key is to find it here. + m.schemaWarnings.push_back(std::format( + "[target.{}] has unsupported key '{}' (ignored). Supported keys: " + "cxx_runtime, linkage, runner, sysroot, toolchain. " + "Per-role contracts go in [build].cxx_runtime's table form.", + triple, key)); + } +``` + +Update the comment block above it: the sweep now covers scalars and arrays; sub-tables remain excluded for the reason already recorded. + +`types.cppm:763`: comment on `deps` — "resolved for this host at load; see `resolve_host_value`". + +- [ ] **Step 4: Run to verify they pass** + +Run: `mcpp build && mcpp test unit/test_manifest` +Expected: `ok.` Also run the existing `EveryBuildKeyTheParserReadsIsAccepted`-style negative controls; they must stay green. + +- [ ] **Step 5: Commit** + +```bash +git add modules/manifest tests/unit/test_manifest.cpp +git commit -m "manifest: [xlings] values per host platform; the target sweep reports array typos and lists runner (#544)" +``` + +--- + +### Task 4: Runner lookup and messages + +**Files:** +- Create: `src/build/runner_lookup.cppm` (module `mcpp.build.runner_lookup`) +- Modify: `src/build/prepare.cppm:637-720` (`BuildContext` gains `xlingsDepBinDirs`), and the site after `fillXpkgDirs` (`:4549`) that has both `ctx` and the payload directories +- Test: `tests/unit/test_runner_lookup.cpp` + +**Interfaces:** +- Produces: + +```cpp +export namespace mcpp::build::runner_lookup { +struct Lookup { + std::optional program; // absolute, executable + std::vector searched; // in order, for the message +}; +// argv0 absolute or containing a separator: taken as-is when executable. +// Otherwise: /argv0, then each PATH entry (pathEnv split on +// the platform separator), first executable regular file wins. +Lookup locate(std::string_view argv0, + std::span depBinDirs, + std::string_view pathEnv); +enum class SpawnClass { Unloadable, Other }; +// ENOEXEC (and EBADARCH on macOS, when defined) → Unloadable; anything else → Other. +SpawnClass classify(int spawnErrno); +std::string not_found_message(std::string_view triple, std::string_view argv0, + std::span searched); +std::string spawn_failed_message(std::string_view program, int spawnErrno); +std::string unrunnable_message(std::string_view triple, const std::filesystem::path& artifact, + int spawnErrno); +} +``` + +`BuildContext::xlingsDepBinDirs` is `std::vector`: `/bin` for each installed `[xlings] deps` payload of the runtime-owner manifest, declaration order. + +- [ ] **Step 1: Write the failing tests** (`tests/unit/test_runner_lookup.cpp`) + +```cpp +#include +#include +#include +import std; +import mcpp.build.runner_lookup; +using namespace mcpp::build::runner_lookup; + +#if !defined(_WIN32) +namespace { +std::filesystem::path make_exe(const std::filesystem::path& dir, std::string_view name) { + std::filesystem::create_directories(dir); + auto p = dir / name; + { std::ofstream o(p); o << "#!/bin/sh\nexit 0\n"; } + std::filesystem::permissions(p, std::filesystem::perms::owner_all); + return p; +} +} + +TEST(RunnerLookup, PayloadBinBeatsPath) { + auto root = std::filesystem::temp_directory_path() / "mcpp-runner-lookup"; + std::filesystem::remove_all(root); + auto inPayload = make_exe(root / "payload" / "bin", "qemu-x"); + auto onPath = make_exe(root / "path", "qemu-x"); + std::vector bins{root / "payload" / "bin"}; + auto l = locate("qemu-x", bins, (root / "path").string()); + ASSERT_TRUE(l.program.has_value()); + EXPECT_EQ(*l.program, inPayload); +} + +TEST(RunnerLookup, PathIsSearchedAfterPayloads) { + auto root = std::filesystem::temp_directory_path() / "mcpp-runner-lookup2"; + std::filesystem::remove_all(root); + auto onPath = make_exe(root / "path", "qemu-y"); + std::vector bins{root / "payload" / "bin"}; // absent dir + auto l = locate("qemu-y", bins, (root / "path").string()); + ASSERT_TRUE(l.program.has_value()); + EXPECT_EQ(*l.program, onPath); + ASSERT_EQ(l.searched.size(), 2u); + EXPECT_EQ(l.searched[0], root / "payload" / "bin"); +} + +TEST(RunnerLookup, NotFoundListsEveryDirectorySearched) { + auto root = std::filesystem::temp_directory_path() / "mcpp-runner-lookup3"; + std::vector bins{root / "a" / "bin"}; + auto l = locate("nope", bins, (root / "p1").string() + ":" + (root / "p2").string()); + EXPECT_FALSE(l.program.has_value()); + ASSERT_EQ(l.searched.size(), 3u); + auto msg = not_found_message("aarch64-linux-musl", "nope", l.searched); + EXPECT_NE(msg.find("'nope'"), std::string::npos) << msg; + EXPECT_NE(msg.find((root / "p2").string()), std::string::npos) << msg; + EXPECT_NE(msg.find("[xlings]"), std::string::npos) << msg; +} + +TEST(RunnerLookup, AbsoluteArgv0IsTakenAsIs) { + auto root = std::filesystem::temp_directory_path() / "mcpp-runner-lookup4"; + auto abs = make_exe(root, "runner.sh"); + auto l = locate(abs.string(), {}, ""); + ASSERT_TRUE(l.program.has_value()); + EXPECT_EQ(*l.program, abs); +} + +TEST(RunnerLookup, ClassifiesENOEXECAsUnloadable) { + EXPECT_EQ(classify(ENOEXEC), SpawnClass::Unloadable); + EXPECT_EQ(classify(EACCES), SpawnClass::Other); + EXPECT_EQ(classify(ENOENT), SpawnClass::Other); +} + +TEST(RunnerLookup, UnrunnableMessageNamesKernelAnswerTripleAndKey) { + auto msg = unrunnable_message("aarch64-linux-musl", "/x/bin/app", ENOEXEC); + EXPECT_NE(msg.find("Exec format error"), std::string::npos) << msg; + EXPECT_NE(msg.find("aarch64-linux-musl"), std::string::npos); + EXPECT_NE(msg.find("[target.aarch64-linux-musl]"), std::string::npos); + EXPECT_NE(msg.find("runner = [\"qemu-aarch64-static\"]"), std::string::npos) << msg; + EXPECT_NE(msg.find("--no-runner"), std::string::npos); +} +#endif +``` + +- [ ] **Step 2: Run to verify they fail** + +Run: `mcpp build && mcpp test unit/test_runner_lookup` +Expected: build error, module `mcpp.build.runner_lookup` does not exist. + +- [ ] **Step 3: Implement `src/build/runner_lookup.cppm`** + +```cpp +// mcpp.build.runner_lookup — where the runner's program is, and what to say +// when it is not. +// +// The lookup is mcpp's own rather than posix_spawnp's for one measured reason: +// a bare name on PATH resolves to an xvm shim, and the shim answers for the +// current subos rather than for the package (e2e 130 in CI; `python3` on this +// machine). The payload's bin/ is the binary itself, so it is searched first. +// Doing the lookup here also splits "not found anywhere" (decided before any +// spawn) from a spawn-time ENOENT, which can then only mean "found, but its +// interpreter or loader is missing". +export module mcpp.build.runner_lookup; +import std; +import mcpp.platform; + +export namespace mcpp::build::runner_lookup { + +struct Lookup { + std::optional program; + std::vector searched; +}; + +namespace detail { +inline bool executable_file(const std::filesystem::path& p) { + std::error_code ec; + if (!std::filesystem::is_regular_file(p, ec)) return false; + auto perms = std::filesystem::status(p, ec).permissions(); + using P = std::filesystem::perms; + return (perms & (P::owner_exec | P::group_exec | P::others_exec)) != P::none; +} +} + +inline Lookup locate(std::string_view argv0, + std::span depBinDirs, + std::string_view pathEnv) +{ + Lookup out; + std::filesystem::path a0(argv0); + if (a0.is_absolute() || argv0.find('/') != std::string_view::npos + || argv0.find('\\') != std::string_view::npos) { + if (detail::executable_file(a0)) out.program = std::filesystem::absolute(a0); + out.searched.push_back(a0.parent_path()); + return out; + } + for (auto const& d : depBinDirs) { + out.searched.push_back(d); + if (auto c = d / a0; detail::executable_file(c)) { out.program = c; return out; } + } + constexpr char sep = mcpp::platform::is_windows ? ';' : ':'; + for (auto part : std::views::split(pathEnv, sep)) { + std::filesystem::path d(std::string_view(part.begin(), part.end())); + if (d.empty()) continue; + out.searched.push_back(d); + if (auto c = d / a0; detail::executable_file(c)) { out.program = c; return out; } + } + return out; +} + +enum class SpawnClass { Unloadable, Other }; + +inline SpawnClass classify(int e) { + if (e == ENOEXEC) return SpawnClass::Unloadable; +#if defined(EBADARCH) + if (e == EBADARCH) return SpawnClass::Unloadable; +#endif + return SpawnClass::Other; +} + +inline std::string errno_text(int e) { return std::generic_category().message(e); } + +inline std::string not_found_message(std::string_view triple, std::string_view argv0, + std::span searched) { + std::string dirs; + for (auto const& d : searched) dirs += "\n " + d.string(); + return std::format( + "runner '{}' for '{}' was not found. Searched:{}\n" + " Declare the package that provides it under [xlings] deps, or " + "install it on PATH.\n" + " Pass --no-runner to execute the artifact directly on this host.", + argv0, triple, dirs); +} + +inline std::string spawn_failed_message(std::string_view program, int e) { + return std::format("runner '{}' could not be started: {} (error {})", + program, errno_text(e), e); +} + +inline std::string unrunnable_message(std::string_view triple, + const std::filesystem::path& artifact, int e) { + return std::format( + "this host cannot execute '{}': {} (error {}).\n" + " The artifact was built for '{}'. Declare how to run it here:\n" + "\n" + " [target.{}]\n" + " runner = [\"qemu-aarch64-static\"]\n" + "\n" + " The artifact path is appended, or substituted for `{{}}` if the " + "template contains it.\n" + " A host that can execute it directly may pass --no-runner.", + artifact.string(), errno_text(e), e, triple, triple); +} + +} // namespace +``` + +Windows: `ENOEXEC` exists in `` on MSVC; `is_windows` is in `mcpp.platform`. Keep the module free of POSIX headers. + +`BuildContext` in `prepare.cppm`: add after `depSourceRoots`: + +```cpp + // `/bin` of every installed `[xlings] deps` payload of the + // runtime-owner manifest, in declaration order. Read by choose_runner's + // lookup (mcpp.build.runner_lookup) so a runner may name a program the + // project declared, without writing the payload's home-and-version path + // into the manifest. Computed by the same resolution fillXpkgDirs uses for + // build programs; a declared-but-absent payload contributes nothing. + std::vector xlingsDepBinDirs; +``` + +Fill it next to `fillXpkgDirs` (`:4549`), from `runtimeOwnerManifest.xlings.deps` through `xpkg_payload`, appending `*dir / "bin"`. Confirm `ctx` is in scope at that point; if the lambda's enclosing function does not own `ctx`, compute the vector in the same function that builds the `BuildContext` and hand it in. + +- [ ] **Step 4: Run to verify they pass** + +Run: `mcpp build && mcpp test unit/test_runner_lookup` +Expected: `ok.` + +- [ ] **Step 5: Commit** + +```bash +git add src/build/runner_lookup.cppm src/build/prepare.cppm tests/unit/test_runner_lookup.cpp +git commit -m "build: runner lookup through declared payloads then PATH, with typed messages (#544)" +``` + +--- + +### Task 5: `choose_runner` on every target; `mcpp run`; `--no-runner`; fast-path marker + +**Files:** +- Modify: `src/build/execute.cppm:53-95` (cache entry), `:150-215` (reader), `:280-300` (writer), `:420-451` (`choose_runner`), `:1100-1190` (`try_fast_run`), `:1191-1320` (`build_run_target`) +- Modify: `src/cli/cmd_build.cppm:194-218` (`cmd_run`), `src/cli.cppm:383-395` (option) +- Test: e2e in Task 7 (the unit-testable part is in Task 4) + +**Interfaces:** +- `choose_runner(const BuildContext&, bool noRunner)`: `tmpl` filled for any target; `freestanding` unchanged in meaning; new field `bool ignored` when `noRunner` dropped a declared template (so the caller prints one note). +- `build_run_target(..., const std::string& target_triple = {}, bool no_runner = false)`. +- `BuildCacheEntry::runnerDeclared` (bool, line `runner=0|1`, written always; absent on old caches → false). + +- [ ] **Step 1: `choose_runner`** + +```cpp +struct RunnerChoice { + std::vector tmpl; + bool freestanding = false; // an EMPTY tmpl is fatal when true + bool fromManifest = false; + bool ignored = false; // --no-runner dropped a declared template +}; + +RunnerChoice choose_runner(const BuildContext& ctx, bool noRunner = false) { + RunnerChoice c; + if (auto ft = mcpp::toolchain::triple::parse(ctx.tc.targetTriple)) + c.freestanding = ft->is_freestanding(); + // Both producers, for every target. The freestanding predicate used to + // gate this read, which is how a runner declared under a hosted cross + // triple was validated, documented and never consulted (#544). It now + // decides one thing only: whether an absent runner is fatal. + c.tmpl = ctx.manifest.buildConfig.runner; + if (auto it = ctx.manifest.targetOverrides.find(ctx.tc.targetTriple); + it != ctx.manifest.targetOverrides.end() && !it->second.runner.empty()) { + c.tmpl = it->second.runner; + c.fromManifest = !ctx.manifest.buildConfig.runner.empty(); + } + if (noRunner && !c.tmpl.empty()) { c.tmpl.clear(); c.ignored = true; } + return c; +} +``` + +`--no-runner` on a freestanding target with the template dropped: the existing `no_runner_message` path fires, which is the correct answer (there is nothing to execute directly). + +- [ ] **Step 2: `mcpp run` launch (replace `execute.cppm:1264-1319`)** + +```cpp + const auto choice = choose_runner(*ctx, no_runner); + if (choice.ignored) + mcpp::ui::info("note", std::format( + "--no-runner: ignoring the runner declared for {}", ctx->tc.targetTriple)); + if (choice.fromManifest) + mcpp::ui::info("note", std::format( + "[target.{}].runner overrides the runner a dependency supplied", + ctx->tc.targetTriple)); + if (choice.freestanding && choice.tmpl.empty()) { + std::println(stderr, "error: {}", + mcpp::freestanding::no_runner_message(ctx->tc.targetTriple)); + return 2; + } + if (!choice.tmpl.empty()) { + auto tmpl = choice.tmpl; + const char* pathEnv = std::getenv("PATH"); + auto found = mcpp::build::runner_lookup::locate( + tmpl.front(), ctx->xlingsDepBinDirs, pathEnv ? pathEnv : ""); + if (!found.program) { + std::println(stderr, "error: {}", mcpp::build::runner_lookup::not_found_message( + ctx->tc.targetTriple, tmpl.front(), found.searched)); + return 2; + } + tmpl.front() = found.program->string(); + argv = mcpp::freestanding::expand(tmpl, exe); + for (auto& a : passthrough) argv.push_back(a); + mcpp::ui::status("Running", std::format("`{} … {}`", choice.tmpl.front(), + mcpp::ui::shorten_path(exe, pathCtx))); + } else { + argv.push_back(exe.string()); + for (auto& a : passthrough) argv.push_back(a); + mcpp::ui::status("Running", std::format("`{}`", mcpp::ui::shorten_path(exe, pathCtx))); + } + std::println(""); std::fflush(stdout); + ... childEnv as today ... + int spawnErr = 0; + int rc = mcpp::platform::process::run_exec(argv, childEnv, &spawnErr); + if (spawnErr != 0) { + using namespace mcpp::build::runner_lookup; + if (!choice.tmpl.empty()) + std::println(stderr, "error: {}", spawn_failed_message(argv.front(), spawnErr)); + else if (classify(spawnErr) == SpawnClass::Unloadable) + std::println(stderr, "error: {}", unrunnable_message(ctx->tc.targetTriple, exe, spawnErr)); + else + std::println(stderr, "error: {}", spawn_failed_message(exe.string(), spawnErr)); + return 2; + } + return rc == 0 ? 0 : 1; +``` + +Add `import mcpp.build.runner_lookup;` to `execute.cppm`. + +- [ ] **Step 3: Fast path** + +`BuildCacheEntry`: add `bool runnerDeclared = false;` with a comment: "a runner declared for this target makes the fast path a miss, because the fast path has no manifest to read the template from and must not execute the artifact bare when the prepare path would not". Writer: `f << "runner=" << (e.runnerDeclared ? 1 : 0) << '\n';` after the `subos=` line. Reader: optional line `runner=`. Where the cache entry is populated (search `runTargets =` assignments in `run_build_plan`/`prepare` — `git grep -n "\.runTargets" src/build`), set `e.runnerDeclared = !choose_runner(ctx).tmpl.empty();`. In `try_fast_run`, after `if (!chosen) return std::nullopt;`: `if (match->runnerDeclared) return std::nullopt;`. Also make `try_fast_run` pass `&spawnErr` to `run_exec` and print `unrunnable_message`/`spawn_failed_message` on failure with exit 2, so the two doors behave the same. + +- [ ] **Step 4: CLI** + +`src/cli.cppm` run subcommand: `.option(cl::Option("no-runner").help("Execute the artifact directly, ignoring any [target.].runner (a host that can run it natively)"))`. Same option on `test`. `cmd_run`: `bool no_runner = parsed.is_flag_set("no-runner");` and pass it through. Help line at `src/cli.cppm:62` for `test`: append `--no-runner`. + +- [ ] **Step 5: Build and smoke by hand** + +```bash +mcpp build +cd $(mktemp -d) && $MCPP new probe >/dev/null && cd probe +printf '\n[target.x86_64-linux-gnu]\nrunner = ["/bin/sh", "-c", "echo RUNNER-SAW \"$@\"", "wrap"]\n' >> mcpp.toml +$MCPP run # expect RUNNER-SAW +$MCPP run # second run: fast path must still go through the runner (marker) +$MCPP run --no-runner # expect the note and the program's own output +``` + +(Use the triple `$MCPP toolchain list` reports for the host if it is not `x86_64-linux-gnu`.) + +- [ ] **Step 6: Commit** + +```bash +git add src/build/execute.cppm src/cli/cmd_build.cppm src/cli.cppm +git commit -m "run: honour the declared runner on every target; --no-runner; typed launch failures (#544)" +``` + +--- + +### Task 6: `mcpp test` — `NotRun`, one reason, exit 2 + +**Files:** +- Modify: `src/build/execute.cppm:1324-1345` (`TestOptions` gains `bool noRunner`), `:1346-1356` (`TestRunSummary` gains `int notRun`, `std::string notRunReason`), `:1527-1560` (`TestResult::St::NotRun`, JSON), `:1711-1800` (workers), `:1855-1905` (argv build), `:1905-1950` (summary) +- Modify: `src/cli/cmd_build.cppm:221-262` (`to.noRunner = parsed.is_flag_set("no-runner")`), and the `--workspace` aggregation at `:280-395` (count `notRun` into the workspace summary line as `; N not run`) + +- [ ] **Step 1: State and JSON** + +```cpp +enum class St { Pass, CompileFail, RunFail, NotRun } status; +... +const char* st = r.status == TestResult::St::Pass ? "pass" + : r.status == TestResult::St::CompileFail ? "compile_fail" + : r.status == TestResult::St::NotRun ? "not_run" + : "run_fail"; +// record gains `"reason":""` after `timed_out`. +``` + +Add `std::string reason;` to `TestResult`. + +- [ ] **Step 2: Runner resolution once, before Pass 2** + +Where argv is built per test (`:1855-1875`), replace the freestanding gate: + +```cpp + const auto choice = choose_runner(*ctx, testOpts.noRunner); + // resolved once per invocation, outside the loop: hoist `choice` and + // the lookup above the `for (auto& lu : ...)` loop. +``` + +Above the loop: + +```cpp + const auto choice = choose_runner(*ctx, testOpts.noRunner); + if (choice.ignored && !json) mcpp::ui::info("note", "--no-runner: ignoring the declared runner"); + if (choice.freestanding && choice.tmpl.empty()) { ...unchanged error, return 2... } + std::vector runnerTmpl = choice.tmpl; + std::string invocationNotRunReason; // non-empty ⇒ nothing is spawned + if (!runnerTmpl.empty()) { + const char* pathEnv = std::getenv("PATH"); + auto found = mcpp::build::runner_lookup::locate( + runnerTmpl.front(), ctx->xlingsDepBinDirs, pathEnv ? pathEnv : ""); + if (found.program) runnerTmpl.front() = found.program->string(); + else invocationNotRunReason = mcpp::build::runner_lookup::not_found_message( + ctx->tc.targetTriple, runnerTmpl.front(), found.searched); + } +``` + +Per test: `argv = runnerTmpl.empty() ? {exe} : expand(runnerTmpl, exe)`. + +- [ ] **Step 3: Workers** + +Add `std::atomic hostCannotRun{false}; std::string hostCannotRunReason; std::mutex reasonMutex;` next to `next`. In the worker, before spawning: + +```cpp + if (!invocationNotRunReason.empty() || hostCannotRun.load()) { + std::scoped_lock lock(reportMutex); + if (!json) mcpp::ui::plain(std::format("{} ... not run", r.name)); + results.push_back({r.name, TestResult::St::NotRun, 0, {}, {}, 0, false, + invocationNotRunReason.empty() ? hostCannotRunReason + : invocationNotRunReason}); + emit_json(results.back()); + continue; + } + int spawnErr = 0; + ... capture_exec_deadline(..., &spawnErr) / run_exec_deadline(..., &spawnErr) ... + if (spawnErr != 0) { + std::string reason = runnerTmpl.empty() + ? (runner_lookup::classify(spawnErr) == SpawnClass::Unloadable + ? std::format("this host cannot execute {} artifacts: {}", + ctx->tc.targetTriple, runner_lookup::errno_text(spawnErr)) + : runner_lookup::spawn_failed_message(r.argv.front(), spawnErr)) + : runner_lookup::spawn_failed_message(r.argv.front(), spawnErr); + { + std::scoped_lock lock(reportMutex); + if (!hostCannotRun.exchange(true)) { + hostCannotRunReason = reason; + if (!json) mcpp::ui::warning(reason); // printed once + } + if (!json) mcpp::ui::plain(std::format("{} ... not run", r.name)); + results.push_back({r.name, TestResult::St::NotRun, 0, {}, {}, ms, false, reason}); + emit_json(results.back()); + } + continue; + } +``` + +Keep `TestResult`'s field order consistent with these brace-initialisers (add `reason` last). + +- [ ] **Step 4: Summary** + +```cpp + int passed = 0, failed = 0, notRun = 0; + std::string notRunReason; + for (auto& r : results) { + if (r.status == TestResult::St::Pass) ++passed; + else if (r.status == TestResult::St::NotRun) { ++notRun; if (notRunReason.empty()) notRunReason = r.reason; } + else { ++failed; failures.push_back(r.name); } + } + summary.notRun = notRun; summary.notRunReason = notRunReason; + // JSON summary: add "not_run":N,"not_run_reason":"…" + ... + const int rc = failed ? 1 : (notRun ? 2 : 0); + if (json) { ...print...; return rc; } + std::println(""); + auto counts = std::format("{} passed; {} failed", passed, failed); + if (notRun) counts += std::format("; {} not run ({})", notRun, first_line(notRunReason)); + if (rc == 0) { mcpp::ui::status("test result", std::format("ok. {}; finished in {}", counts, timing)); return 0; } + mcpp::ui::error(std::format("test result: {}. {}; finished in {}", + failed ? "FAILED" : "NOT RUN", counts, timing)); + if (failed) { println failures block as today } + return rc; +``` + +`first_line` = the reason up to its first `\n`. Workspace fan-out in `cmd_build.cppm`: sum `sum.notRun`, print `; N not run` in the workspace line when non-zero, and make the workspace rc 2 when any member returned 2 and none returned 1. + +- [ ] **Step 5: Build, then hand-check with a two-test project and the patched-artifact technique from Task 7** + +- [ ] **Step 6: Commit** + +```bash +git add src/build/execute.cppm src/cli/cmd_build.cppm +git commit -m "test: unrunnable artifacts are reported not-run once, with the reason, and exit 2 (#544)" +``` + +--- + +### Task 7: e2e 330 + +**Files:** +- Create: `tests/e2e/330_runner_hosted_targets.sh` + +- [ ] **Step 1: Write the script** + +```bash +#!/usr/bin/env bash +# requires: unix-shell +# 330_runner_hosted_targets.sh — [target.].runner on a hosted target, +# and what `mcpp run` / `mcpp test` say when the host cannot execute an artifact. +# +# Issue #544. Design: .agents/docs/2026-09-02-runner-beyond-baremetal-design.md §11. +# +# No emulator is needed and none is used: the runner is a shell script that +# records its argv, and the "unloadable artifact" is a real host binary whose +# ELF e_machine is patched to 0xffff after the build, which no binfmt entry and +# no native loader accepts (ENOEXEC on any Linux host, binfmt_misc or not). +# `requires: unix-shell` and nothing else: a `requires: gcc` or `requires: llvm` +# guard skips on both CI shards, and a skip exits 0. +set -uo pipefail +TMP=$(mktemp -d); trap "rm -rf $TMP" EXIT; cd "$TMP" +fail() { echo "FAIL: $*"; exit 1; } +MCPP="${MCPP:?set MCPP to the binary under test}" + +HOST=$("$MCPP" toolchain list --format json 2>/dev/null | sed -n 's/.*"host":"\([^"]*\)".*/\1/p' | head -1) +[[ -n "$HOST" ]] || HOST=$(uname -m)-linux-gnu # fallback: adjust if toolchain list has no host field +case "$(uname -s)" in Darwin) echo "SKIP: e_machine patching is ELF-only; the runner half runs below";; esac + +"$MCPP" new app >/dev/null || fail "mcpp new" +cd app +mkdir -p tests +cat > src/main.cpp <<'EOF' +#include +int main() { std::puts("ARTIFACT-RAN"); return 0; } +EOF +cat > tests/one.cpp <<'EOF' +int main() { return 0; } +EOF +cat > "$TMP/runner.sh" <<'EOF' +#!/bin/sh +printf '%s\n' "$@" >> "$RUNNER_LOG" +exec "$@" +EOF +chmod +x "$TMP/runner.sh" +export RUNNER_LOG="$TMP/runner.log" + +# ── 1. a declared runner is used on a hosted target, on both doors ───────── +printf '\n[target.%s]\nrunner = ["%s"]\n' "$HOST" "$TMP/runner.sh" >> mcpp.toml +out=$("$MCPP" run 2>&1) || fail "run through runner: $out" +grep -q "ARTIFACT-RAN" <<<"$out" || fail "artifact output missing: $out" +[[ -s "$RUNNER_LOG" ]] || fail "runner was not invoked" +exe=$(head -1 "$RUNNER_LOG") +[[ "$exe" == */bin/app ]] || fail "runner argv[0] is not the artifact: $exe" +: > "$RUNNER_LOG" +out=$("$MCPP" run 2>&1) || fail "second run (fast path): $out" +[[ -s "$RUNNER_LOG" ]] || fail "fast path bypassed the runner" + +# ── 2. --no-runner executes directly and says so ─────────────────────────── +: > "$RUNNER_LOG" +out=$("$MCPP" run --no-runner 2>&1) || fail "--no-runner: $out" +grep -q "ARTIFACT-RAN" <<<"$out" || fail "--no-runner lost the program output: $out" +grep -q "no-runner" <<<"$out" || fail "--no-runner printed no note: $out" +[[ ! -s "$RUNNER_LOG" ]] || fail "--no-runner still invoked the runner" + +# ── 3. runner not found: error names the program, the search list, exit 2 ── +sed -i "s|runner = \[.*\]|runner = [\"mcpp-e2e-no-such-runner\"]|" mcpp.toml +out=$("$MCPP" run 2>&1); rc=$? +[[ $rc -eq 2 ]] || fail "missing runner: exit $rc, want 2: $out" +grep -q "runner 'mcpp-e2e-no-such-runner'" <<<"$out" || fail "missing runner not named: $out" +grep -q "Searched:" <<<"$out" || fail "search list missing: $out" +grep -q "ARTIFACT-RAN" <<<"$out" && fail "artifact ran without its runner" +out=$("$MCPP" test 2>&1); rc=$? +[[ $rc -eq 2 ]] || fail "test with missing runner: exit $rc, want 2: $out" +grep -qE "NOT RUN\. 0 passed; 0 failed; 1 not run \(runner 'mcpp-e2e-no-such-runner'" <<<"$out" \ + || fail "test summary: $out" + +# ── 4. no runner, unloadable artifact (ELF only) ─────────────────────────── +if [[ "$(uname -s)" == Linux ]]; then + sed -i "/^\[target\.$HOST\]/,+1d" mcpp.toml + "$MCPP" build >/dev/null 2>&1 || fail "rebuild without runner" + bin=$(ls target/*/*/bin/app | head -1) + printf '\xff\xff' | dd of="$bin" bs=1 seek=18 conv=notrunc status=none + out=$("$MCPP" run 2>&1); rc=$? + [[ $rc -eq 2 ]] || fail "unrunnable: exit $rc, want 2: $out" + grep -q "Exec format error" <<<"$out" || fail "kernel answer missing: $out" + grep -q "\[target.$HOST\]" <<<"$out" || fail "paste-able key missing: $out" + grep -q "runner = \[" <<<"$out" || fail "runner example missing: $out" + # the artifact must still be the patched one, or the criterion measured a rebuild + od -An -tx1 -j18 -N2 "$bin" | grep -q "ff ff" || fail "artifact was rebuilt under the run" + # mcpp test: one test and two tests, streaming and capturing paths + cat > tests/two.cpp <<'EOF' +int main() { return 0; } +EOF + for n in 1 2; do + [[ $n -eq 1 ]] && rm -f tests/two.cpp + "$MCPP" build >/dev/null 2>&1 + "$MCPP" test --list >/dev/null 2>&1 + # build the test binaries once so they exist to patch: run the suite (it runs them), then patch and rerun + "$MCPP" test >/dev/null 2>&1 || true + for t in target/*/*/tests/bin/*; do [[ -f "$t" ]] && printf '\xff\xff' | dd of="$t" bs=1 seek=18 conv=notrunc status=none; done + out=$("$MCPP" test 2>&1); rc=$? + [[ $rc -eq 2 ]] || fail "test unrunnable ($n): exit $rc, want 2: $out" + grep -qE "NOT RUN\. 0 passed; 0 failed; $n not run \(this host cannot execute" <<<"$out" \ + || fail "test summary ($n): $out" + [[ $(grep -c "cannot execute" <<<"$out") -eq 2 ]] || fail "reason printed other than once + summary ($n): $out" + jout=$("$MCPP" test --message-format json 2>/dev/null) + [[ $(grep -c '"status":"not_run"' <<<"$jout") -eq $n ]] || fail "json not_run count ($n): $jout" + grep -q "\"not_run\":$n" <<<"$jout" || fail "json summary not_run ($n): $jout" + done +fi + +# ── 5. array-valued typo is reported and runner is in the list ───────────── +printf '\n[target.%s]\nrunnerX = ["x"]\n' "$HOST" >> mcpp.toml +out=$("$MCPP" build 2>&1) +grep -q "unsupported key 'runnerX'" <<<"$out" || fail "array typo not reported: $out" +grep -q "Supported keys: cxx_runtime, linkage, runner, sysroot, toolchain" <<<"$out" || fail "runner not listed: $out" +echo "PASS: 330_runner_hosted_targets" +``` + +Adjust the test-binary location (`target/*/*/tests/bin/*`) to where `mcpp test` actually places them (check `ls target/*/*/` after a run) and the host-triple discovery to what `mcpp toolchain list` prints; both are facts to read from the built engine, not to guess. + +- [ ] **Step 2: Run against the unfixed engine to record the failure, then against the fixed one** + +```bash +MCPP=$(which mcpp) bash tests/e2e/330_runner_hosted_targets.sh # released 2026.8.17.1: must FAIL at section 1 +MCPP=$PWD/target/*/*/bin/mcpp bash tests/e2e/330_runner_hosted_targets.sh # must PASS +``` + +Also re-run the neighbours: `130`, `131`, `132`, `178`. + +- [ ] **Step 3: Commit** + +```bash +git add tests/e2e/330_runner_hosted_targets.sh +git commit -m "e2e: runner on hosted targets, not-run reporting, --no-runner (#544)" +``` + +--- + +### Task 8: Documentation (English and Chinese) + +**Files:** +- Modify: `docs/05-mcpp-toml.md` (§2.7.1 table row; new §2.7.3 "Running an artifact this host cannot execute" placed after §2.7.2 with the two forms and the `[xlings] deps` consequence; §2.13 per-platform values), `docs/13-baremetal.md` (cross-reference under "An absent runner"), `docs/15-openkal-cross.md` (hosted example after `:190`), `docs/17-the-project-environment.md` (per-platform values), `docs/11-machine-output.md` (new §7 kind `mcpp.test` describing the NDJSON record and summary, with `not_run`) +- Modify: the five `docs/zh/` twins with the same content in Chinese. + +- [ ] **Step 1: Write the English sections** (register per `.agents/skills/mcpp-docs-style/SKILL.md`; declarative headings; state the `[xlings] deps` consequence in one paragraph; state that `[xlings]` per-platform values are resolved against the host at load, that the keys are `linux`, `macos`, `windows`, `default`, that an entry without a match is not declared on that host, and that an unknown key is an error). +- [ ] **Step 2: Write the Chinese twins.** +- [ ] **Step 3: Run the style check and the doc parity check** + +```bash +bash .github/tools/check_docs_style.sh +``` + +- [ ] **Step 4: Commit** + +```bash +git add docs +git commit -m "docs: runner on hosted targets, not-run reporting, [xlings] per-platform values (#544)" +``` + +--- + +### Task 9: Version bump, PR, CI + +- [ ] **Step 1: Bump** `mcpp.toml` version and `modules/versioning/src/version.cppm` to `2026.9.2.1` (next free `.N` for today; check `git tag -l 'v2026.9.2.*'`). Run `bash .github/tools/check_version_pins.sh`. +- [ ] **Step 2: Full local gates**: `mcpp build && mcpp test` (unit), `MCPP=... bash tests/e2e/330_runner_hosted_targets.sh`, `bash .github/tools/check_modules_wiring.sh`, `bash .github/tools/check_docs_style.sh`, `git diff origin/main --diff-filter=D --name-only` must be empty, `git log --oneline origin/main..HEAD` shows only this branch's commits. +- [ ] **Step 3: Push and open the PR** with a body in English academic register: the defect, the three decisions (no fallback, exit 2, `--no-runner`), the `[xlings]` per-platform addition, the test criteria, and the behaviour changes for existing manifests. Link #544. +- [ ] **Step 4: Watch CI** (`gh pr checks --watch`); fix and push until every required check is green. + +--- + +### Task 10: Self-review, merge, release, ecosystem verification + +- [ ] **Step 1: Self-review** the PR diff against design §0 and §11 (each D1–D6 has a visible implementation or a stated deferral; every criterion row in §11 has an assertion in e2e 330; no criterion is a substring search of a line that could be silent). Use `/code-review` on the PR. +- [ ] **Step 2: Merge** with `gh pr merge --squash --admin`; confirm `gh pr view --json state,mergeCommit`; confirm the run on `origin/main`'s HEAD SHA is green. +- [ ] **Step 3: Release**: `git tag v2026.9.2.1 && git push origin v2026.9.2.1`; watch `release.yml`; if `publish-ecosystem`'s GitCode leg times out, download the missing assets from the verified GitHub release and upload with `/usr/bin/python3 ~/.local/bin/gtc release upload xlings-res/mcpp --tag 2026.9.2.1`, then GET-verify each asset from `https://gitcode.com/xlings-res/mcpp/releases/download/2026.9.2.1/` with `cmp`. Merge the index bump PR on `openxlings/xim-pkgindex` (switch `gh auth` to the account with push, switch back after), confirm `git show origin/main:pkgs/m/mcpp.lua` has `latest` at `2026.9.2.1`, then bump the bootstrap pin in `.xlings.json` through a PR. +- [ ] **Step 4: Sandbox verification**: `xlings update`; `xlings subos create e544-`; inside `xlings subos use e544- --sandbox --cmd "..."`: configure `mcpp self config --mirror CN`, install `mcpp@2026.9.2.1`, then run a base64-inlined script that (a) builds an openkal cross project for `aarch64-linux-musl` with `[xlings] deps = [{ linux = "qemu-user-aarch64" }]` and `runner = ["qemu-aarch64-static"]`, runs it, and asserts the program output; (b) runs `mcpp test --target aarch64-linux-musl` and asserts the summary counts; (c) without the runner asserts the `Exec format error` message and exit 2. Assert the installed version with `mcpp --version` at the start of the script, with a lower bound not an exact match. +- [ ] **Step 5: Record** the sandbox transcript summary in `.agents/docs/2026-09-02-runner-beyond-baremetal-design.md` §13 "Verification" with the exact commands and the lines observed. diff --git a/.agents/docs/2026-09-02-runner-beyond-baremetal-design.md b/.agents/docs/2026-09-02-runner-beyond-baremetal-design.md new file mode 100644 index 00000000..cc2e7dc4 --- /dev/null +++ b/.agents/docs/2026-09-02-runner-beyond-baremetal-design.md @@ -0,0 +1,721 @@ +# The runner beyond bare metal: design + +Date: 2026-09-02. Status: proposal, revised after self-review, awaiting review. +Issue: #544. Code references are against `origin/main` at aeba151 and were +re-verified line by line for this revision. + +## 0. Decisions for review + +The first draft of this document was reviewed against the code, the index, +the kernel, and this repository's recorded history. Six positions changed or +need a decision. Each is stated with what the draft said, what the review +found, and the recommendation the rest of this document is written to. The +alternative is recorded in section 9 so that flipping a decision is an edit to +one section, not a rewrite. + +**D1. A declared runner whose program cannot be started is an error. The +draft fell back to direct execution with a warning.** The fallback contradicts +the draft's own rejection of `when = "cannot-execute"` (section 9): on a host +with binfmt_misc registered, falling back runs the artifact under the kernel's +interpreter without the declared arguments, which is the "runs and misbehaves" +failure the draft used to reject the alternative. The draft's claim that the +fallback is "safe by construction because it reaches `ENOEXEC` immediately" is +false on the very host section 2 was measured on. `ENOENT` from `posix_spawnp` +also means "installed but its interpreter or loader is missing", not only +"absent", so the fallback would run the artifact directly past a broken runner. +Recommendation: no fallback; principle P2 of the draft is withdrawn and +replaced (section 3). The one case the fallback served, a triple that is native +on this host, gets an explicit escape instead (D3). + +**D2. `mcpp test` exits 2 when any test was not run. The draft exited 0.** The +freestanding path already exits 2 for exactly this situation, a target this +host cannot execute with no runner to stand in front of it +(`src/build/execute.cppm:1866`), so the draft gave one situation two exit codes +depending on the triple's `os` field. This repository's own record is that +skip-exits-zero is how `# requires: llvm` tests never ran in CI while every +job stayed green, and CI reads exit codes, not summary lines. Recommendation: +`NotRun` is reported truthfully in the summary and in JSON, and the exit code +is 2 whenever the count is non-zero. `--require-runner` disappears; no +relaxation flag is proposed until someone asks for one with a use. + +**D3. `--no-runner` on `mcpp run` and `mcpp test`.** "This host can execute +the artifact" is a fact about the host, and the manifest has no host axis: +`[target.]` is keyed by target and `[xlings] deps` has no key at all. +A runner written for x86_64 developers is therefore also consulted on an +aarch64 host resolving the same triple, where the emulator is pointless and, +for the index's `qemu-user-aarch64`, uninstallable. The draft handled that host +by guessing (D1). The replacement is a flag the operator on that host passes, +which is the only place the fact is known. Recommendation: adopt; it is small +and it is what keeps P1 true without P2. + +**D4. Provisioning the runner's package through `[xlings] deps` is a shape +for single-host-class projects, and the document must say so.** Two facts the +draft did not have: `[xlings] deps` has no conditional form +(`modules/manifest/src/toml.cppm:1350-1361`), and a package that cannot be +installed on this host is a hard build error, `provisioning [xlings] deps +failed` (`src/build/prepare.cppm:3462-3466`). The index's `qemu-user-aarch64` +declares `archs = {"x86_64"}`. A project that lists it therefore cannot be +built on an aarch64 or macOS host at all, not even with `mcpp build`. The +draft's "two keys that already exist" example is right for a CI matrix on one +host class and wrong as the general recommendation. The resolution rule of +section 4.4, which looks in the declared packages' `bin/` before `PATH`, is +kept because of a third fact: a bare name on `PATH` resolves to an xvm shim +that dispatches against the current subos, and both e2e 130 in CI and this +session (`python3` on `PATH` answering "not installed in this subos") show +that lookup failing with the package installed. Recommendation: keep the rule, +change the recommended shape in the documentation, state the consequence. + +**D5. `runner = ""` is deferred, and the spelling when needed is `[]`.** The +draft's reason for `""` over `[]` does not survive inspection: both spellings +are hard errors on every released mcpp, so neither is "misread" by an older +version, and the draft's distinction was between two error messages. `[]` is +the empty value of the key's own type, as `sysroot = ""` is of its type. The +feature has no producer: no published package emits `mcpp:runner=` for a +hosted triple (the only emitters in the local registry are e2e fixtures 131 +and 132, both freestanding). Recommendation: record the spelling, do not +implement until a dependency supplies a runner for a hosted target. + +**D6. Scope beyond the issue.** Three items the design adds are listed so they +can be cut individually: the unknown-key sweep learning about array-valued +keys (issue ask 3, second half); `docs/05` moving `runner` out from under +"Bare metal"; and `docs/11` gaining the `mcpp test --message-format json` +record, which the machine-output contract page does not document today at +all, so the new `not_run` status would otherwise be a change to an +undocumented stream. Recommendation: include all three; the third is the one +most reasonably deferred to its own change. + +**D7. `[xlings]` values can be given per host platform, in the form xlings +already defines.** Decided on review of D4 (2026-09-02). xlings' `.xlings.json` +resolves a `workspace` value that is an object keyed by platform against the +host it runs on (`src/core/xvm/db.cppm:375-421` in the xlings repository: +keys `linux`, `macosx`, `windows`, `default`; no match and no default means +the entry is absent on that host). mcpp's `[xlings.workspace]` parser accepted +only strings and dropped a table value in silence (`toml.cppm:1356-1358`), so +the form the section claims to mirror 1:1 was not mirrored. `deps` has no +xlings-side semantics (mcpp reads the list and calls `install_packages`), so +its per-platform form is mcpp's to define, and it takes the same one: an +entry is a string or a `{ = "" }` table. Both keys are +resolved against the host when the manifest is loaded, which keeps every +downstream reader (the provisioning pass, its stamp, the build-program +hand-off) on a flat list. Platform keys are `linux`, `macos`, `windows` and +`default`, with `macosx` accepted as xlings' own spelling; an unknown key is a +hard error rather than a dropped entry. The axis is the host OS only; a +package that exists for the OS but not the architecture (`qemu-user-aarch64` +on aarch64 Linux) still fails provisioning loudly, and `--no-runner` remains +the answer on that host. Section 4.4's recommended CI form becomes: + +```toml +[xlings] +deps = [{ linux = "qemu-user-aarch64" }] + +[target.aarch64-linux-musl] +runner = ["qemu-aarch64-static"] +``` + +## 1. What the defect is + +`[target.].runner` is parsed, type-checked and validated for every +triple a manifest names, and consulted for exactly one class of them. +`choose_runner` returns an empty template before reading either producer unless +the target is freestanding: + +```cpp +// src/build/execute.cppm:435 +RunnerChoice choose_runner(const BuildContext& ctx) { + RunnerChoice c; + auto ft = mcpp::toolchain::triple::parse(ctx.tc.targetTriple); + if (!ft || !ft->is_freestanding()) return c; // :438 + ... + c.tmpl = ctx.manifest.buildConfig.runner; // :445 build.mcpp channel + if (auto it = ctx.manifest.targetOverrides.find(ctx.tc.targetTriple); + it != ctx.manifest.targetOverrides.end() && !it->second.runner.empty()) { + c.tmpl = it->second.runner; // :448 manifest channel +``` + +`is_freestanding()` is `os == "none"` (`modules/toolchain-model/src/triple.cppm:132`). +Both channels a runner can arrive by, the manifest key and a build program's +`mcpp:runner=`, are therefore bypassed before either is read, on every hosted +target. `mcpp run` gates on the same predicate at `execute.cppm:1272-1273` and +`mcpp test` at `:1861-1862`. + +The manifest layer, by contrast, treats the key as load-bearing on every triple. +`runner = []` is a hard error naming the triple (`modules/manifest/src/toml.cppm:1910-1913`), +so the value's shape is checked where the value will never be read. Two smaller +inconsistencies accompany it. The unsupported-key warning for `[target.]` +lists `cxx_runtime, linkage, sysroot, toolchain` and omits `runner`; the sweep +skips arrays on purpose (`toml.cppm:1941-1946`, whose comment records that an +earlier version reported `runner` as "unsupported (ignored)" while honouring +it), so an array-valued typo is silent. And `docs/05-mcpp-toml.md` documents +the key only inside section 2.7.2, "Bare metal", while its own section 2.7.1 +table says an exact triple "also carries `toolchain` / `linkage`" and not +`runner`. + +Two failures compound the scope question and are independent of it. + +1. **The two launch paths disagree about a spawn failure.** `run_exec` + converts any `posix_spawnp` failure into a bare 127 with nothing printed + (`modules/platform/src/process.cppm:566-567`). Its sibling `capture_exec` + formats the same condition through `spawn_failure(argv.front(), sp)` + (`:616-620`, helper at `:392`), and that text lands in the test's + `runOutput`. `mcpp test` captures when there is more than one test or when + `--message-format json` is passed, and streams otherwise + (`execute.cppm:1713`). The reporter had exactly one test, so they saw the + silent path; a second test file would have produced + `posix_spawnp('…') failed (error 8): Exec format error` in the failures + block. The symptom in #544 depends on the number of tests, which is the + shape this repository records as "the loud failure has a silent twin". +2. **The bounded launcher states the invariant that this violates, and then + defeats it.** On POSIX it returns `supported = false` when the child could + not be spawned (`modules/platform/src/unix/bounded_process.cppm:227-230`), + and its declaration says why: "'Could not spawn' and 'ran and failed' must + not share an exit code, so the caller falls back rather than reporting a + failure" (`:60-62`). Both callers fall back by spawning again: + `run_exec_deadline` into `run_exec` (`process.cppm:746`) and + `capture_exec_deadline` into `capture_exec` (`:857`). The first attempt's + errno is discarded, the child is spawned twice, and on the untimed path the + second attempt swallows what the first one knew. + +## 2. Measured facts + +The design rests on what the kernel reports, so the readings were taken before +the design was written and repeated for this revision. The probe spawns four +targets through `posix_spawnp` on the host used for this work (x86_64 Linux, +glibc, binfmt_misc populated by qemu-user): + +| Case | `posix_spawn` result | Child ran | +|---|---|---| +| ELF with `e_machine = EM_AARCH64`, binfmt_misc registered | returns 0, child exits 255 | yes, under qemu | +| ELF with `e_machine = 0xffff`, no binfmt entry matches | returns 8, `ENOEXEC` | no | +| A program name absent from `PATH` | returns 2, `ENOENT` | no | +| A host binary | returns 0, child exits 0 | yes | + +The artifact in the first row was synthesised by patching `e_machine` on a host +binary, so the 255 is qemu's own error code rather than a program's; that +coincidence is itself a measurement, and section 10 returns to it. + +Three consequences follow, and each is load-bearing below. + +**The host's ability to execute a foreign-ISA artifact is not a function of the +triple.** This machine registers several dozen architectures under +`/proc/sys/fs/binfmt_misc/`; `qemu-aarch64` carries flags `PO`. On it, +`mcpp run --target aarch64-linux-musl` starts successfully. On the machine that +reported #544 the same command yields `Exec format error`. The two hosts +disagree about the same target, and the deciding state is a kernel table that +neither mcpp nor the manifest can observe. The reverse asymmetry also holds: a +dynamically linked `aarch64-linux-musl` artifact need not run on an aarch64 +glibc host. Equality of architecture is neither necessary nor sufficient. + +**A failed spawn has no side effects.** glibc reports the exec failure through +the return value and the child never runs. The design below does not attempt +a second launch, but the property still matters: a spawn refused with +`ENOEXEC` leaves nothing behind, so a `mcpp test` invocation that stops after +the first refusal has not half-run anything. + +**`ENOEXEC` and `ENOENT` are distinguishable at the point of failure, and +`ENOENT` is ambiguous.** `ENOEXEC` means the kernel refused to load the +artifact. `ENOENT` means the program named in `argv[0]` could not be started, +which covers three situations: it is not on the search path; it is a script +whose `#!` interpreter is missing; it is a dynamic executable whose ELF +interpreter is missing. Only the first is "not installed". The design +separates the first from the other two by doing its own lookup before +spawning (section 4.4), and reports the other two verbatim. + +Four further facts were established for this revision, and each one changed +the design: + +- **The index package is `qemu-user-aarch64`, and its program is + `qemu-aarch64-static`** (`pkgs/q/qemu-user-aarch64.lua`: `programs = + {"qemu-aarch64-static"}`, installed to `/bin/`, registered as an + xvm shim of the same name). The draft's `deps = ["qemu-user"]` and + `runner = ["qemu-aarch64"]` named a package and a program the index does + not have. The package is `archs = {"x86_64"}` by design: on an aarch64 host + an aarch64 binary just runs. +- **A bare name on `PATH` resolves to a shim that answers for the current + subos, not for where the package is installed.** In this session, `python3` + on `PATH` answered `python3 is not installed in this subos (_) — installed + elsewhere 3.13.12`. e2e 130 records the same for `qemu-system-riscv64` in + CI (`tests/e2e/130_freestanding_riscv_build_and_run.sh:44-51`) and works + around it by resolving the emulator to an absolute path under + `xpkgs/*-x-qemu-riscv/*/bin`. +- **`[xlings] deps` is unconditional and its provisioning failure is fatal.** + The key has no `cfg()` or per-triple form (`toml.cppm:1350-1361`). A package + xlings cannot resolve or install ends the build with `provisioning [xlings] + deps failed: …` (`prepare.cppm:3462-3466`); that path was made to read the + result in #531 precisely so that a declaration cannot look accepted and do + nothing. +- **Tests run concurrently.** `run_tests_now` uses + `min(runJobs, N)` workers when capturing (`execute.cppm:1711-1717`). A rule + phrased as "the first failure decides for the rest" has to be written for + workers that may already be past the check. + +## 3. Principles + +Three rules generate the whole design. They are stated first so that later +sections can be checked against them rather than argued individually. + +**P1. mcpp does not predict whether the host can execute an artifact.** It +either does what the project declared, or it attempts execution and reports what +the kernel answered. No table maps a triple to an executability verdict. + +**P2. Host-local facts are stated by the operator on that host.** The manifest +describes the project. Whether this machine can execute a given artifact, and +whether the runner the project named is wanted here, are facts about the +machine, and the operator on it is the only party that knows them. mcpp +provides a way to state them (`--no-runner`, section 7) and does not infer +them. + +The draft's P2, "a configuration describing an environment this machine lacks +must not disable a capability this machine has", is withdrawn. It licensed a +guess, and the guess was wrong on the host it was measured on (D1). + +**P3. Failing loudly outranks succeeding differently.** Where two defaults each +have a failure mode, the default is the one whose failure is an error message +rather than a program that runs and behaves incorrectly. + +## 4. Structure + +### 4.1 One read point, unchanged + +`choose_runner` remains the single function both `mcpp run` and `mcpp test` +consult. The comment above it records why (`execute.cppm:420-425`): deriving the +answer twice is a shape this codebase has paid for in #233, #240, #242 and #344. +This design changes what the function decides, not how many places decide it. + +The freestanding predicate moves out of the resolution path and into a single +remaining role: whether an absent runner is fatal before any spawn is +attempted. `RunnerChoice::freestanding` keeps that meaning; `RunnerChoice::tmpl` +is filled for every target that declares one. + +### 4.2 Resolution order + +For the resolved target, in order: + +1. **A runner declared for this target.** The project's + `[target.].runner` beats a dependency's `mcpp:runner=`, which is the + existing precedence (`execute.cppm:446-450`) and the existing note when the + consumer overrides (`:1286-1289`). +2. **No declaration.** Execute the artifact directly. + +The per-triple key is already the scoping mechanism for "this runner belongs to +that cross target": a runner written under `[target.aarch64-linux-musl]` is not +found when the host target is resolved, because the lookup is +`targetOverrides.find(ctx.tc.targetTriple)` and the key is stored canonicalised +(`toml.cppm:1825-1830, :1960`). No new key is needed to express applicability +along the target axis. + +What the key cannot express is the host axis, and this is the limit D3 and D4 +respond to: the same triple is foreign on one host and native on another, and +`[target.]` reads the same on both. + +The channel that has no triple scope is `BuildConfig::runner` +(`modules/manifest/src/types.cppm:458`), which a dependency's build program +writes. Its merge site says it "supplies a runner for this target" +(`prepare.cppm:7240-7256`) while the storage is per-build. One build resolves +one target, so the two coincide in practice; the exposure is a dependency that +should not be in the host target's graph emitting a runner. That is a +dependency-scoping question and is out of scope here, recorded in section 10. + +### 4.3 Launch + +``` +--no-runner passed? +├── yes → spawn the artifact directly (declared runner ignored, one note) +└── no + declared runner? + ├── yes → resolve argv[0] (section 4.4) + │ ├── not found on any search path → ERROR: names the program, the paths + │ │ searched, the [xlings] deps hint; exit 2 + │ └── found → spawn + │ ├── started → done; its exit code is the verdict + │ └── any spawn error → ERROR: program and errno verbatim; exit 2 + └── no → spawn the artifact directly + ├── started → done + ├── ENOEXEC → UNRUNNABLE: what the kernel said, the triple, + │ and the runner key to write; exit 2 + └── other spawn error → ERROR: program and errno verbatim; exit 2 +``` + +There is no second spawn on any branch. A runner that cannot be started is +reported, not worked around (D1); an artifact the kernel refuses is reported +with the key that would have changed the outcome. `EACCES` on either is a +permission problem, not an absence, and is reported as itself. + +Freestanding keeps its stronger contract: with no runner declared, `mcpp run` +and `mcpp test` fail with `no_runner_message` before any spawn +(`src/freestanding/runner.cppm:57`). There is no direct-execution branch, and +its absence is provable rather than measured. The hosted UNRUNNABLE message is +a sibling of that one with different wording: it reports what the kernel +answered (`Exec format error`) rather than asserting why, because on a hosted +triple mcpp does not know whether the refusal is a foreign ISA or a file that +is not an executable at all, and the example it prints is a user-mode emulator +(`qemu-aarch64-static`) rather than a system one. + +Exit codes. `mcpp run` today folds every non-zero result, including the +program's own exit status, into 1 (`execute.cppm:1319`); that fold is +pre-existing and not changed here, but "could not start" is distinguished from +"ran and failed" by exiting 2, the code the freestanding no-runner path already +uses. The distinction the bounded launcher's comment demands (section 1) is +thereby made at the caller rather than lost at the callee. + +### 4.4 Resolving the runner's program + +`BuildConfig::runner` exists because the value is machine-specific: the emulator +lives in a package payload whose path carries a home and a version, "so only a +`build.mcpp` can compute it" (`types.cppm:446-450`). That reasoning is one step +too strong. The engine already computes exactly this mapping for build +programs: `fillXpkgDirs` (`prepare.cppm:4549-4566`) walks the manifest's +`[xlings] deps`, resolves each to its payload directory through +`xlings::paths::xpkg_payload`, and hands the result down as `MCPP_XPKG_*` +environment variables that `xpkg_dir()` reads back (`src/build/hostprogram.cppm:322`). +A build program needs `xpkg_dir()` because it was the only caller with a reason +to ask, not because it is the only caller that can. + +Therefore: + +> The runner's `argv[0]`, when it is not an absolute path, is resolved by mcpp +> against the `bin/` directory of each payload this project declared under +> `[xlings] deps`, in declaration order, and then against `PATH`. If no +> candidate is an executable file, the runner is reported as not found before +> any spawn is attempted. + +The rule earns its place for one reason, and it is not the one the draft gave. +A bare name on `PATH` reaches an xvm shim, and the shim answers for the current +subos rather than for the package (section 2). The payload's `bin/` is the +binary itself. Looking there first is what makes a declared package usable +from a runner without the project author writing a home-and-version path into +the manifest, which is the thing `BuildConfig::runner`'s comment says a static +manifest cannot do. + +Doing the lookup in mcpp has a second effect: "not found anywhere" is decided +before `posix_spawnp`, so a spawn-time `ENOENT` can only mean the program was +found and its interpreter or loader was not. The two messages differ, and +neither guesses. + +**The recommended shape, and its two forms.** The general form is one key and +a tool the operator has installed by whatever means, including +`xlings install qemu-user-aarch64`: + +```toml +[target.aarch64-linux-musl] +runner = ["qemu-aarch64-static"] +``` + +The CI form adds the package declaration so that a fresh runner provisions the +emulator on first use, through the same pass, stamp and `--offline` refusal +that every other `[xlings] deps` entry already goes through +(`prepare.cppm:3242` onwards): + +```toml +[xlings] +deps = ["qemu-user-aarch64"] + +[target.aarch64-linux-musl] +runner = ["qemu-aarch64-static"] +``` + +The CI form has a consequence the documentation must state next to it: +`[xlings] deps` is provisioned on every host that builds the project, without +condition, and a package the host cannot install is a hard build error. With +`qemu-user-aarch64` being x86_64-only, the CI form makes the project +unbuildable on an aarch64 or macOS host until the line is removed. That is the +correct behaviour for the key (a declaration that silently does nothing was +#531) and the wrong shape for a project built on more than one host class. The +general form has no such consequence: a runner whose program is absent fails +only `mcpp run` and `mcpp test`, only on that host, with a message naming the +program, and `--no-runner` runs the artifact directly where the host can. + +This keeps one declaration site for packages. `[xlings] deps` +(`types.cppm:763`, parsed at `toml.cppm:1351`) remains the only place that +answers "which packages does this project need". An attribute on `runner` +carrying a package reference was considered and rejected for this reason; see +section 9. + +Scope is deliberately narrow: this changes how mcpp locates the runner it was +told about. It does not change `PATH` as seen by the program under test. +`mcpp test` already prepends the sandbox's `subos/default/bin` to the child +environment (`execute.cppm:1886-1897`) and `mcpp run` does not +(`:1309-1314`); that asymmetry predates this work and should be settled on its +own merits, not folded in here. + +## 5. `runner = []`: deferred + +There is currently no way to state "ignore the runner a dependency supplied and +execute this artifact directly". Omitting the key inherits the dependency's +value; `runner = []` is a hard error. + +The feature is deferred because it has no producer. No published package emits +`mcpp:runner=` for a hosted triple; the only emitters in the local registry +are the fixtures of e2e 131 and 132, both freestanding. The parse-time error +for `[]` stays as it is until a dependency supplies a hosted runner. + +The spelling is decided now so that the first producer does not reopen it: +`[]`, not `""`. `sysroot` established the shape (`toml.cppm:1877-1889`, +documented at `docs/05-mcpp-toml.md:1185`): an absent key and a present-and-empty +key are different answers, and the empty answer is spelled as the empty value +of the key's own type. Both `[]` and `""` are hard errors on every released +mcpp, so neither degrades better than the other on an older version; `""` +would add a second type to the key for no gain. Under a freestanding triple +`[]` stays rejected at parse time, because on such a target direct execution +is provably not available. + +## 6. `mcpp test`: not-run is a reading, not a failure, and not a success + +A cross-built test suite on a machine that cannot execute its artifacts has not +failed. Reporting `FAIL (exit 127)` states that the test ran and returned 127, +which is false, and is indistinguishable from a missing program. It has not +passed either, and an exit code that says it did would be read as one. + +`TestResult::St` gains a fourth state alongside `Pass`, `CompileFail` and +`RunFail` (`execute.cppm:1529`): `NotRun`, carrying a reason. The rules: + +- **One reason, N results.** An `ENOEXEC` on any test's artifact establishes + that this host cannot execute artifacts of this target for this invocation; + a runner that cannot be found or started establishes that no test can be + run through it. Either sets an invocation-wide flag that workers check + before each spawn. Workers already past the check may fail the same way; + that is harmless (a refused spawn has no side effects) and each such result + is `NotRun`, not `RunFail`. The reason is printed once, when it is first + established, and appears again in the summary. +- **The count is always in the summary when non-zero**, at the same visual + weight as failures, with the denominator, and in the wording e2e 178 already + asserts for the workspace-timeout case: `test result: NOT RUN. 0 passed; + 0 failed; 5 not run (this host cannot execute aarch64-linux-musl artifacts: + Exec format error); finished in 0.41s`. When failures and not-run tests + coexist the prefix is `FAILED` and both counts are listed. +- **`--message-format json` carries `not_run` as its own status** next to + `pass`, `compile_fail` and `run_fail` (`:1543-1545`), with `exit_code` 0 and a + `reason` string; the summary record gains `not_run` and `not_run_reason`. + The record's other fields keep their types. +- **The exit code is 2 whenever the not-run count is non-zero** (D2). It is + the code the freestanding path already returns for the same situation, and + it is distinct from 1, which continues to mean that a test ran and failed. + +## 7. `--no-runner` + +One flag, accepted by `mcpp run` and `mcpp test`: ignore any declared runner +and execute the artifact directly. It prints one note naming the runner it +ignored, so a transcript shows the deviation. + +It exists for the host the manifest cannot describe (D3): a triple that is +native here although the project declares an emulator for it. It is also the +honest answer for a wrapper an operator does not want on this run +(`valgrind`, `sudo -E`, `ssh board`). Without it, that operator edits the +manifest, and the edit is host-shaped. + +The draft's `--require-runner` is withdrawn. Its two roles were to make +`NotRun` fatal, which is now the default (D2), and to refuse the `ENOENT` +fallback, which no longer exists (D1). A per-declaration `required = true` +attribute is likewise moot. + +## 8. The five axes + +**Structure.** One read point is preserved. The freestanding predicate keeps a +single, smaller job. Package declaration stays in `[xlings] deps`; execution +stays in `runner`; the coupling between them is an engine resolution rule with +no representation in the schema, so no consumer of "the declared package set" +gains a second site to read. The launcher layers stop disagreeing about spawn +failures because the errno is carried up (`DeadlineRun` gains the spawn +errno; callers stop re-spawning), and the two `mcpp test` launch paths report +the same condition the same way. + +**Stability.** Every decision is either a value the project wrote, a flag the +operator passed, or an answer the kernel gave. Nothing is inferred from host +state that mcpp models itself, which is the class of defect the N x N +cross-build work found nine instances of. There is no branch that runs a +program in a way other than the one declared. + +**Cross-platform.** Linux is measured (section 2). The design reads the +platform's exec decision rather than reproducing it, so binfmt_misc, Rosetta, +WOW64 and ARM64EC need no cases in mcpp. Two legs are unmeasured and must not be +implemented from inference: macOS `posix_spawn` may report a wrong-architecture +Mach-O as `EBADARCH` rather than `ENOEXEC`, and on Windows `run_exec` still goes +through `std::system` (`process.cppm:571-576`), where the error is lost in the +shell before it can be typed. The residual `TODO(launcher-unify)` at +`process.cppm:549` names the prerequisite. Until both are measured, the Windows +leg reports the failure it can observe and does not claim to distinguish the +two situations, and the e2e criteria of section 11 are POSIX-gated for that +reason and no other. + +**Compatibility.** The behavioural change needs no new manifest key, so a +manifest written for it loads on older mcpp and degrades correctly there: the +older version does not consult the runner and reports the failure it has. The +new flag is a CLI addition. A table form for `runner` was rejected partly on +this axis: `runner` is currently required to be an array (`toml.cppm:1896-1899`), +so a package shipping `runner = { ... }` does not lose a key on an older mcpp, +it fails to load at all, the failure mode recorded in #359. + +Two behaviour changes affect existing manifests and are stated rather than +assumed harmless. A `runner` already declared on a hosted triple stops being +inert and starts being used; the documented population is small, because the key +is documented only under "Bare metal", and a runner that was inert and is now +missing becomes an error on that host rather than a silent no-op. And a +`mcpp test` run that reported `FAIL` for unrunnable artifacts now reports +`NotRun`; its exit code stays non-zero, so no CI job changes colour, but its +text and JSON change. + +**Simplicity.** The common case is one key that already exists and a tool on +`PATH`. The mechanism a user must understand is one sentence: mcpp uses the +runner you declared, and tells you what the kernel said when there is none. + +## 9. Rejected alternatives + +**A host-capability oracle.** Deciding before spawning, from the triple plus +probes of binfmt_misc, Rosetta and similar. Rejected on measurement: this host +runs `aarch64-linux-musl` artifacts directly and the host that filed #544 does +not, so any triple-shaped verdict is wrong on one of them. Every entry in such a +table is a host-shaped branch. It would also be wrong in the other direction for +a dynamically linked artifact on a same-architecture host with a different C +library. + +**`when = "cannot-execute"` as the default.** Consulting the runner only after +direct execution fails. It expresses "this runner is for cross targets" +directly, but as a default it inverts P3: on a host with binfmt registered, +direct execution starts, the declared runner is skipped, and any argument it +carried (a `-L` sysroot, a machine model) is silently dropped. That failure is a +program that runs and misbehaves; the alternative failure is an error message. + +**Falling back to direct execution when the runner's program is missing** +(the draft's section 4.3). Rejected for the same reason as the previous entry, +which it reproduced with a warning attached: on a binfmt host the artifact +runs under a different interpreter with different arguments. It also required +a second spawn, and it read `ENOENT` as "absent" when the errno also covers a +present runner with a missing interpreter or loader. What it was for, a host +where the triple is native, is served by `--no-runner`, which states the fact +instead of guessing it. + +**`runner = { command = [...], package = "xim:qemu-user-aarch64" }`.** Rejected +because it opens a second package-declaration site. Every consumer of "what +does this project need installed", the offline refusal message, the first-use +install list, any enumeration, would have to read both, and the ones that +forget do not fail; they answer a different question quietly. Section 4.4 +obtains the same capability as a resolution rule with one declaration site and +no schema change. Its cost, that `[xlings] deps` is unconditional, is stated +there rather than hidden by a second site that would be conditional only by +accident. + +**`runner = ""` as the override spelling.** Rejected in favour of `[]`; see +section 5. + +**`--require-runner`.** Withdrawn; both of its roles are the default now. See +section 7. + +**A relaxation flag that lets `mcpp test` exit 0 with tests not run.** Not +proposed. A matrix job that builds a target it cannot run should call +`mcpp build` for that target; a job that calls `mcpp test` has asked a +question and should not be told "yes" when the answer is "not established". +If a use appears, the flag is additive. + +## 10. Open questions + +1. **macOS.** Measure `posix_spawn` against a wrong-architecture Mach-O. + `EBADARCH` and `ENOEXEC` must both map to UNRUNNABLE, and the mapping must be + measured, not assumed. +2. **Windows.** Determine whether the launcher must be moved onto + `CreateProcess` before the failure can be typed at all. Wine is not evidence + for Windows. +3. **`run` and `test` disagree about the child's `PATH`** (`execute.cppm:1309-1314` + versus `:1886-1897`). Deliberate or drift, to be answered separately. +4. **`BuildConfig::runner` has no triple scope.** Whether a dependency that + should not be in a host target's graph can supply a runner for it is a + dependency-scoping question. +5. **A runner's own failure is not typeable.** When qemu exits 255 because it + could not start, mcpp cannot tell that from the program exiting 255. The + measurement in section 2 shows this concretely. This is a documented limit, + not a defect to fix by guessing. +6. **The manifest has no host axis.** `--no-runner` is a per-invocation + statement; a developer on an aarch64 host types it every time. Whether the + manifest should gain a host predicate, or whether a host-local + configuration file should carry "this host executes `` directly", + is the durable answer, and it is larger than this issue. +7. **`mcpp run` folds the program's exit status into 1** (`execute.cppm:1319`). + `cargo run` propagates it. Whether mcpp should is unrelated to the runner + and is noted because this work touches the line. + +## 11. Test criteria + +The criteria below are written to survive the two failure modes this repository +has recorded most often: an assertion that passes because it never ran, and an +assertion whose negative reading is also its silent reading. + +**No `# requires:` guard on the new e2e.** Both CI shards lack llvm, and a +`# requires:` skip exits 0, so a criterion behind one never runs in CI. The +hosted-runner criteria therefore use a runner that is a shell script in the +project directory, recording `"$@"` to a file and then executing the artifact. +The script runs on every POSIX shard, needs no emulator, and lets the +assertion compare the exact argv mcpp launched against the artifact path. + +**The unrunnable path must be exercised on any host, including one with +binfmt_misc.** The probe in section 2 supplies the technique: after +`mcpp build`, patch `e_machine` of the produced binary to `0xffff`, which +matches no binfmt entry and no native loader, so `posix_spawnp` returns +`ENOEXEC` deterministically. The criterion asserts, after the run, that the +artifact still carries the patched machine, so a rebuild between the patch +and the run is a loud failure of the test rather than a silent pass of the +host binary. + +**Each criterion carries a denominator.** For `mcpp test`, assert the full +summary line including every count, not the presence of the words "not run". +For the runner path, assert the argv the script recorded equals the artifact +path, not that a message mentioning the runner appeared. + +**Both `mcpp test` launch paths get the same criterion.** The single-test +invocation streams and the two-test invocation captures (section 1), and today +they report a spawn failure differently. Each criterion below that involves +`mcpp test` is run once with one test file and once with two, and the +assertion on the reason text is the same in both. + +| Criterion | Setup | Assertion | +|---|---|---| +| Declared runner is used on a hosted target | script runner, host target | recorded argv equals the artifact path; exit code is the artifact's | +| Runner program not found | runner names an absent program | message names the program and the paths searched; exit 2; the artifact was not executed | +| Runner found under a declared payload, not on `PATH` | `[xlings] deps` names a package whose `bin/` holds the script; `PATH` does not | recorded argv proves the payload copy ran | +| No runner, unloadable artifact, `mcpp run` | `e_machine` patched | message contains `Exec format error`, the triple and the paste-able key; exit 2 | +| No runner, unloadable artifact, `mcpp test` | `e_machine` patched, one and two tests | `NOT RUN. 0 passed; 0 failed; N not run (…)` with N equal to the test count; exit 2; JSON records carry `not_run` | +| `--no-runner` | script runner declared | the script's record file is absent; the artifact ran; one note names the ignored runner | +| Freestanding contract unchanged | `os = none`, no runner | `no_runner_message`, exit 2, no spawn | +| Array-valued typo is reported | `runnerX = ["x"]` under `[target.]` | warning names `runnerX` and lists `runner` among supported keys | + +**Three of these must fail before the fix and pass after.** The two +unrunnable cases produce exit 127 with no output today on the streaming path, +so a criterion asserting only a non-zero exit would pass against the unfixed +engine; the assertions are on the message text and the exit code together. +The declared-runner case fails today because the script is never invoked, so +its record file is absent, which is the assertion's negative reading and must +therefore be paired with the positive one (the artifact's own output appeared +through the script). + +**A unit test states the invariant the layers disagree about.** `run_exec` +must not return 127 for a spawn failure without reporting it; `DeadlineRun` +must carry the spawn errno when `supported` is false; and neither +`run_exec_deadline` nor `capture_exec_deadline` may spawn a second time after +the bounded launcher reports a spawn failure. Section 1 shows these are one +defect seen from two sides; a test that covers only the untimed path leaves +the doubled spawn in place. + +**What is not exercised.** Provisioning `qemu-user-aarch64` through +`[xlings] deps` is not an e2e criterion: it needs network, an x86_64 host, +and a registry, and the provisioning pass has its own coverage from #531. The +payload-`bin/` rule is exercised with a locally staged package directory +instead (third row). + +## 12. Implementation surface + +Listed so the change can be sized and split. Each line is one concern. + +- `src/build/execute.cppm`: `choose_runner` reads both producers for every + target; `mcpp run` and `mcpp test` take `--no-runner`; `TestResult::St::NotRun`, + the invocation-wide flag, the summary line and the JSON record; exit codes. +- `modules/platform/src/process.cppm` and `unix/bounded_process.cppm`: + `DeadlineRun` carries the spawn errno; `run_exec` reports the failure it + drops today; callers stop re-spawning; the runner lookup helper. +- `src/freestanding/runner.cppm`: a hosted sibling of `no_runner_message`. +- `modules/manifest/src/toml.cppm`: the sweep learns a known-arrays list + (`runner`) and prints both lists; nothing else in the parser changes. +- `docs/05-mcpp-toml.md` (2.7.1 gains `runner`; 2.7.2 keeps the bare-metal + example; a hosted-cross subsection with the two forms and the `[xlings] + deps` consequence), `docs/13-baremetal.md` (cross-reference), + `docs/15-openkal-cross.md` (the example at :185 is bare-metal; add the + hosted one), `docs/11-machine-output.md` (the `mcpp test` record, D6), and + each one's `docs/zh/` twin, which CI enforces. +- `tests/e2e/`: one new script covering the table in section 11; + `tests/unit/`: the launcher invariant. diff --git a/modules/platform/src/process.cppm b/modules/platform/src/process.cppm index 657e49fa..db751579 100644 --- a/modules/platform/src/process.cppm +++ b/modules/platform/src/process.cppm @@ -82,17 +82,33 @@ RunResult capture_with_env( // calling process environment is never mutated, so a target's loader vars // (LD_LIBRARY_PATH) cannot poison mcpp itself or any sibling host process. // Returns a platform-normalized exit code, or 127 if exec fails. +// +// ⚠️ A SPAWN FAILURE IS REPORTED EXACTLY ONCE AND NEVER DROPPED (#544). Every +// launcher below follows one rule: when the child could not be started it +// returns 127 and either (a) stores the errno in `*spawn_error` and prints +// nothing, because a caller that asked for the errno owns the report, or +// (b) with `spawn_error == nullptr`, reports it itself — on stderr here, into +// `output` for the capturing variants. Before this rule `run_exec` turned +// every posix_spawnp failure into a bare 127: `mcpp run` on an artifact the +// kernel refused printed "Running …", a blank line, and exited 1. +// +// `*spawn_error` is 0 whenever the child was spawned, whatever it then did. +// On the residual Windows std::system branch the launch cannot be told apart +// from the child, so it stays 0 and the shell's own message is what is seen. int run_exec(const std::vector& argv, - const std::vector>& extraEnv = {}); + const std::vector>& extraEnv = {}, + int* spawn_error = nullptr); // Same as run_exec but captures stdout AND stderr combined (replaces the old // `… 2>&1` redirect) into RunResult::output. Required because the only consumer // (ninja fast-path) parses error text — which ninja writes to stderr — via // is_stale_ninja_failure / filter_ninja_output. No shell → no quoting/injection. +// `spawn_error`: the same contract as run_exec's. RunResult capture_exec( const std::vector& argv, const std::vector>& extraEnv = {}, - std::string_view cwd = {}); + std::string_view cwd = {}, + int* spawn_error = nullptr); // Deadline variants: kill the child once `deadline` elapses and set // *timed_out. A zero deadline means no limit. @@ -107,10 +123,13 @@ RunResult capture_exec( // through to the unbounded launcher, so every timeout knob (`mcpp test // --timeout`, `--build-timeout`, `[build] build_program_timeout`) was a silent // no-op there — set, reported nowhere, and doing nothing. +// `spawn_error`: run_exec's contract. A refused spawn is typed here from the +// bounded launcher's own attempt; there is no second spawn. int run_exec_deadline(const std::vector& argv, const std::vector>& extraEnv, std::chrono::milliseconds deadline, - bool* timed_out); + bool* timed_out, + int* spawn_error = nullptr); // Run one host-shell command with inherited stdio, a working directory and a // real deadline. POSIX uses /bin/sh; Windows uses cmd.exe. This is for @@ -172,12 +191,15 @@ void stop_background(const BackgroundCommand& child, void guard_background_on_signal(const BackgroundCommand& child); void clear_background_guard(); +// `spawn_error`: run_exec's contract; with it null a refused spawn is +// formatted into `output`, as capture_exec does. RunResult capture_exec_deadline( const std::vector& argv, const std::vector>& extraEnv, std::chrono::milliseconds deadline, bool* timed_out, - std::string_view cwd = {}); + std::string_view cwd = {}, + int* spawn_error = nullptr); // Run `command` silently (discard stdout/stderr). // On POSIX, stdin is automatically redirected from /dev/null. @@ -335,6 +357,21 @@ int normalize_exit_code(int rc) { #endif } +// The one sentence every launcher prints for a refused spawn. Platform-neutral +// on purpose: the bounded launchers on both platforms now hand their spawn +// error up (DeadlineRun::spawn_error), and the wrappers that receive it are +// compiled everywhere. `error` is an errno on POSIX and a GetLastError() value +// on Windows; the category below renders each in its own vocabulary. +std::string spawn_failure(std::string_view program, int error) { +#if defined(_WIN32) + return std::format("CreateProcess('{}') failed (error {}): {}\n", + program, error, std::system_category().message(error)); +#else + return std::format("posix_spawnp('{}') failed (error {}): {}\n", + program, error, std::generic_category().message(error)); +#endif +} + #if defined(__linux__) || defined(__APPLE__) // Portable accessor for the host environment block. On Apple, `environ` is // only linkable from executables (not dylibs), so _NSGetEnviron() is the @@ -388,11 +425,6 @@ std::vector merged_environ( } return out; } - -std::string spawn_failure(std::string_view program, int error) { - return std::format("posix_spawnp('{}') failed (error {}): {}\n", - program, error, std::generic_category().message(error)); -} #else // Build a shell command line from an argv vector (Windows + residual non-POSIX // fallback only; Linux/macOS exec directly, #248). EVERY token is shell-quoted, @@ -550,8 +582,10 @@ int run_passthrough(std::string_view command, std::string* output) { // child-only env isolation, move it onto a CreateProcess/_spawn equivalent and // delete the residual shell branch below. int run_exec(const std::vector& argv, - const std::vector>& extraEnv) + const std::vector>& extraEnv, + int* spawn_error) { + if (spawn_error) *spawn_error = 0; if (argv.empty()) return 127; #if defined(__linux__) || defined(__APPLE__) auto envStore = merged_environ(extraEnv); @@ -563,8 +597,15 @@ int run_exec(const std::vector& argv, cargv.push_back(nullptr); pid_t pid = 0; - if (::posix_spawnp(&pid, cargv[0], nullptr, nullptr, cargv.data(), envp.data()) != 0) - return 127; // spawn failed (e.g. program not found) + if (int sp = ::posix_spawnp(&pid, cargv[0], nullptr, nullptr, cargv.data(), envp.data()); + sp != 0) { + // Reported once: by the caller when it asked for the errno, here + // otherwise. Never dropped — the errno in hand at this line is the + // whole difference between "Exec format error" and a blank line. + if (spawn_error) *spawn_error = sp; + else std::fputs(spawn_failure(argv.front(), sp).c_str(), stderr); + return 127; + } int status = 0; while (::waitpid(pid, &status, 0) < 0) { /* EINTR retry */ } return normalize_exit_code(status); @@ -579,9 +620,11 @@ int run_exec(const std::vector& argv, RunResult capture_exec( const std::vector& argv, const std::vector>& extraEnv, - std::string_view cwd) + std::string_view cwd, + int* spawn_error) { RunResult result; + if (spawn_error) *spawn_error = 0; if (argv.empty()) { result.exit_code = 127; return result; } #if defined(__linux__) || defined(__APPLE__) // posix_spawn + a pipe; stdout and stderr both go to the pipe so the @@ -616,7 +659,8 @@ RunResult capture_exec( if (sp != 0) { ::close(fds[0]); result.exit_code = 127; - result.output = spawn_failure(argv.front(), sp); + if (spawn_error) *spawn_error = sp; + else result.output = spawn_failure(argv.front(), sp); return result; } @@ -662,6 +706,7 @@ struct BoundedOutcome { bool supported = false; int exit_code = 0; bool timed_out = false; + int spawn_error = 0; // see DeadlineRun::spawn_error in either launcher std::string output; }; @@ -713,6 +758,7 @@ BoundedOutcome dispatch_bounded( outcome.supported = r.supported; outcome.exit_code = r.exit_code; outcome.timed_out = r.timed_out; + outcome.spawn_error = r.spawn_error; } else { std::vector argvPtrs; argvPtrs.reserve(argv.size()); @@ -723,6 +769,7 @@ BoundedOutcome dispatch_bounded( outcome.supported = r.supported; outcome.exit_code = r.exit_code; outcome.timed_out = r.timed_out; + outcome.spawn_error = r.spawn_error; } return outcome; } @@ -731,10 +778,12 @@ BoundedOutcome dispatch_bounded( int run_exec_deadline(const std::vector& argv, const std::vector>& extraEnv, std::chrono::milliseconds deadline, - bool* timed_out) + bool* timed_out, + int* spawn_error) { if (timed_out) *timed_out = false; - if (deadline.count() <= 0) return run_exec(argv, extraEnv); + if (spawn_error) *spawn_error = 0; + if (deadline.count() <= 0) return run_exec(argv, extraEnv, spawn_error); if (argv.empty()) return 127; // capture=false: identical stdio behaviour to `run_exec` — the child writes @@ -743,7 +792,17 @@ int run_exec_deadline(const std::vector& argv, // undo the observability work that path exists for (and would hide gtest's // colors by making its stdout a pipe). auto r = dispatch_bounded(argv, extraEnv, {}, deadline, /*capture=*/false); - if (!r.supported) return run_exec(argv, extraEnv); + if (!r.supported) { + // Attempted and refused: the errno is in hand, so type it or report + // it here. Spawning again through run_exec — what this did before — + // paid for a second refusal and threw the first errno away (#544). + if (r.spawn_error != 0) { + if (spawn_error) *spawn_error = r.spawn_error; + else std::fputs(spawn_failure(argv.front(), r.spawn_error).c_str(), stderr); + return 127; + } + return run_exec(argv, extraEnv, spawn_error); // no bounded launcher here + } if (timed_out) *timed_out = r.timed_out; return r.exit_code; } @@ -842,19 +901,30 @@ RunResult capture_exec_deadline( const std::vector>& extraEnv, std::chrono::milliseconds deadline, bool* timed_out, - std::string_view cwd) + std::string_view cwd, + int* spawn_error) { if (timed_out) *timed_out = false; - if (deadline.count() <= 0) return capture_exec(argv, extraEnv, cwd); + if (spawn_error) *spawn_error = 0; + if (deadline.count() <= 0) return capture_exec(argv, extraEnv, cwd, spawn_error); RunResult result; if (argv.empty()) { result.exit_code = 127; return result; } auto r = dispatch_bounded(argv, extraEnv, cwd, deadline, /*capture=*/true); // `supported == false` means the child COULD NOT BE SPAWNED — not that it // ran and failed. Reporting those the same way would hide a launcher - // problem behind a child's exit code, so fall back to the untimed path and - // let it produce the real diagnostic. - if (!r.supported) return capture_exec(argv, extraEnv, cwd); + // problem behind a child's exit code. When the launcher attempted the + // spawn, its errno is the diagnostic and there is nothing to retry; only + // a build with no bounded launcher at all falls back to the untimed path. + if (!r.supported) { + if (r.spawn_error != 0) { + result.exit_code = 127; + if (spawn_error) *spawn_error = r.spawn_error; + else result.output = spawn_failure(argv.front(), r.spawn_error); + return result; + } + return capture_exec(argv, extraEnv, cwd, spawn_error); + } result.exit_code = r.exit_code; result.output = std::move(r.output); if (timed_out) *timed_out = r.timed_out; diff --git a/modules/platform/src/unix/bounded_process.cppm b/modules/platform/src/unix/bounded_process.cppm index 5aa0e50c..a8403ec2 100644 --- a/modules/platform/src/unix/bounded_process.cppm +++ b/modules/platform/src/unix/bounded_process.cppm @@ -63,6 +63,14 @@ struct DeadlineRun { bool supported = false; int exit_code = 0; bool timed_out = false; + // Non-zero when `supported` is false BECAUSE the spawn was attempted and + // refused: the errno posix_spawnp returned. Zero with `supported` false + // means this build has no bounded launcher at all. The two need opposite + // treatment one layer up — the first is reported with its errno, the + // second falls back to the unbounded launcher — and before this field + // existed both read the same, so the caller spawned a second time and + // discarded the errno the first attempt had in hand (#544). + int spawn_error = 0; }; using OutputSink = void (*)(void* ctx, const char* data, unsigned long len); @@ -227,7 +235,7 @@ DeadlineRun capture_with_deadline(const char* const* argvEntries, int sp = ::posix_spawnp(&pid, cargv[0], &fa, nullptr, cargv.data(), envp.data()); ::posix_spawn_file_actions_destroy(&fa); if (capture) ::close(fds[1]); - if (sp != 0) { if (capture) ::close(fds[0]); return out; } + if (sp != 0) { out.spawn_error = sp; if (capture) ::close(fds[0]); return out; } // Non-blocking reads so the deadline is still checked while the child is // quiet. A blocking read on a silent, hung child is exactly the hang this diff --git a/modules/platform/src/windows/bounded_process.cppm b/modules/platform/src/windows/bounded_process.cppm index ce930b1a..caf7b415 100644 --- a/modules/platform/src/windows/bounded_process.cppm +++ b/modules/platform/src/windows/bounded_process.cppm @@ -73,6 +73,12 @@ struct DeadlineRun { bool supported = false; int exit_code = 0; bool timed_out = false; + // Non-zero when `supported` is false BECAUSE CreateProcess was attempted + // and refused: its GetLastError() value. Zero with `supported` false means + // this build has no bounded launcher. Same contract as the POSIX peer, + // whose value is an errno; the caller reports the number verbatim and + // does not spawn again (#544). + int spawn_error = 0; }; // Receives stdout+stderr as it arrives. Called on the calling thread only. @@ -305,7 +311,7 @@ DeadlineRun capture_with_deadline(const char* commandLine, envBlock.data(), (cwd && *cwd) ? cwd : nullptr, &si, &pi); - if (!ok) return out; + if (!ok) { out.spawn_error = static_cast(::GetLastError()); return out; } Handle proc; proc.h = pi.hProcess; Handle thread; thread.h = pi.hThread; diff --git a/tests/unit/test_process_run_exec.cpp b/tests/unit/test_process_run_exec.cpp index 2a2a46e0..524d9c27 100644 --- a/tests/unit/test_process_run_exec.cpp +++ b/tests/unit/test_process_run_exec.cpp @@ -1,5 +1,7 @@ #include +#include #include +#include import std; import mcpp.platform.process; @@ -97,6 +99,112 @@ TEST(RunExec, InjectedEnvSurvivesShellChildChain) { EXPECT_EQ(rc, 0); } +// ── spawn failures are typed, reported once, and never retried (#544) ────── +// +// The invariant the bounded launcher's DeadlineRun comment states ("could not +// spawn" and "ran and failed" must not share an exit code) was defeated one +// layer down: run_exec turned every posix_spawnp failure into a bare 127 with +// nothing printed, and both deadline wrappers fell back to it, spawning a +// second time and discarding the first errno. `mcpp run --target +// aarch64-linux-musl` on a host without an emulator printed "Running …", a +// blank line, and exited 1. + +TEST(RunExec, MissingProgramReportsOnStderrWhenCallerDoesNotAsk) { + testing::internal::CaptureStderr(); + int rc = process::run_exec({"/no/such/program/mcpp-run-xyz"}); + auto err = testing::internal::GetCapturedStderr(); + EXPECT_EQ(rc, 127); + EXPECT_NE(err.find("/no/such/program/mcpp-run-xyz"), std::string::npos) << err; + EXPECT_NE(err.find("error 2"), std::string::npos) << err; +} + +TEST(RunExec, MissingProgramTypesErrnoWhenCallerAsks) { + int spawnErr = 0; + testing::internal::CaptureStderr(); + int rc = process::run_exec({"/no/such/program/mcpp-run-xyz"}, {}, &spawnErr); + auto err = testing::internal::GetCapturedStderr(); + EXPECT_EQ(rc, 127); + EXPECT_EQ(spawnErr, ENOENT); + EXPECT_TRUE(err.empty()) << err; // the caller owns the report +} + +TEST(RunExec, UnloadableArtifactIsENOEXEC) { + // An executable file with no loader magic: the kernel refuses it with + // ENOEXEC. posix_spawnp does not retry through /bin/sh the way execvp + // does, so the errno reaches the caller untouched. This is the reading + // `mcpp run` turns into "this host cannot execute the artifact". + auto dir = std::filesystem::temp_directory_path() / "mcpp-enoexec-test"; + std::filesystem::create_directories(dir); + auto f = dir / "not-an-elf"; + { std::ofstream o(f, std::ios::binary); o << "\x7f" "NOT" "\x00\x00\x00\x00"; } + std::filesystem::permissions(f, std::filesystem::perms::owner_all); + int spawnErr = 0; + int rc = process::run_exec({f.string()}, {}, &spawnErr); + EXPECT_EQ(rc, 127); + EXPECT_EQ(spawnErr, ENOEXEC); +} + +TEST(RunExec, SpawnedChildLeavesSpawnErrorZero) { + int spawnErr = -1; + EXPECT_EQ(process::run_exec({"/bin/sh", "-c", "exit 7"}, {}, &spawnErr), 7); + EXPECT_EQ(spawnErr, 0); +} + +TEST(CaptureExec, MissingProgramTypesErrnoWhenCallerAsks) { + int spawnErr = 0; + auto r = process::capture_exec({"/no/such/program/mcpp-capture-typed"}, {}, {}, &spawnErr); + EXPECT_EQ(r.exit_code, 127); + EXPECT_EQ(spawnErr, ENOENT); + EXPECT_TRUE(r.output.empty()) << r.output; +} + +TEST(RunExecDeadline, SpawnFailureIsTypedAndNotRetried) { + int spawnErr = 0; + bool timedOut = true; + testing::internal::CaptureStderr(); + int rc = process::run_exec_deadline({"/no/such/program/mcpp-dl-xyz"}, {}, + std::chrono::milliseconds(5000), &timedOut, + &spawnErr); + auto err = testing::internal::GetCapturedStderr(); + EXPECT_EQ(rc, 127); + EXPECT_EQ(spawnErr, ENOENT); + EXPECT_FALSE(timedOut); + EXPECT_TRUE(err.empty()) << err; +} + +TEST(RunExecDeadline, SpawnFailureIsReportedWhenCallerDoesNotAsk) { + bool timedOut = true; + testing::internal::CaptureStderr(); + int rc = process::run_exec_deadline({"/no/such/program/mcpp-dl-untyped"}, {}, + std::chrono::milliseconds(5000), &timedOut); + auto err = testing::internal::GetCapturedStderr(); + EXPECT_EQ(rc, 127); + EXPECT_FALSE(timedOut); + EXPECT_NE(err.find("mcpp-dl-untyped"), std::string::npos) << err; + EXPECT_NE(err.find("error 2"), std::string::npos) << err; +} + +TEST(CaptureExecDeadline, SpawnFailureIsTypedInOutcome) { + int spawnErr = 0; + bool timedOut = true; + auto r = process::capture_exec_deadline({"/no/such/program/mcpp-cdl-xyz"}, {}, + std::chrono::milliseconds(5000), &timedOut, + {}, &spawnErr); + EXPECT_EQ(r.exit_code, 127); + EXPECT_EQ(spawnErr, ENOENT); + EXPECT_FALSE(timedOut); + EXPECT_TRUE(r.output.empty()) << r.output; +} + +TEST(CaptureExecDeadline, SpawnFailureIsInOutputWhenCallerDoesNotAsk) { + bool timedOut = true; + auto r = process::capture_exec_deadline({"/no/such/program/mcpp-cdl-untyped"}, {}, + std::chrono::milliseconds(5000), &timedOut); + EXPECT_EQ(r.exit_code, 127); + EXPECT_NE(r.output.find("mcpp-cdl-untyped"), std::string::npos) << r.output; + EXPECT_NE(r.output.find("error 2"), std::string::npos) << r.output; +} + #else // _WIN32 TEST(RunExec, WindowsCoveredByIntegration) { From 42fcf96959b3cece6f432006dc2845b32f330f9e Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 2 Sep 2026 11:46:51 +0800 Subject: [PATCH 02/12] manifest: [xlings] values per host platform; the target sweep reports array typos and lists runner (#544) --- modules/manifest/src/toml.cppm | 125 +++++++++++++++++++++++++------- modules/manifest/src/types.cppm | 5 ++ tests/unit/test_manifest.cpp | 108 +++++++++++++++++++++++++++ 3 files changed, 212 insertions(+), 26 deletions(-) diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index 701e8bdf..8226ed59 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -100,6 +100,57 @@ struct LoadContext { bool insideWorkspace = false; }; +// ─── `[xlings]` values per host platform (#544, D7) ───────────────────────── +// +// xlings' own `.xlings.json` lets a `workspace` value be an object keyed by +// platform — `{ "linux": "15.1.0", "default": "22" }` — and resolves it +// against the host it runs on: the host's key wins, `default` is the +// fallback, and no match at all means the entry is absent on that host. +// `[xlings]` claims to mirror that file 1:1, but its parser accepted only +// strings and dropped a table value in silence. `deps` has no xlings-side +// reader (mcpp reads the list and calls `install_packages` itself), so its +// per-platform form is mcpp's to define, and it takes the same one: an entry +// is a string or a `{ = "" }` table. +// +// Resolved HERE, at load, against this host. `[xlings]` describes the +// environment of the machine mcpp runs on, so the host is the right axis, and +// resolving once keeps every downstream reader — the provisioning pass, its +// stamp, the build-program hand-off — on the flat list it already reads. +// +// The keys are mcpp's OS names, `linux` / `macos` / `windows`, plus `default`; +// `macosx` is accepted as xlings' own spelling of `macos`. An unknown key is a +// hard error, not a dropped entry: a typo'd platform that silently declared +// nothing is the shape #531 was filed for. +inline std::string_view host_platform_key() { + if constexpr (mcpp::platform::is_windows) return "windows"; + else if constexpr (mcpp::platform::is_macos) return "macos"; + else return "linux"; +} + +inline std::expected, std::string> +resolve_host_value(const mcpp::libs::toml::Value& v, std::string_view host) { + if (v.is_string()) return std::optional{v.as_string()}; + if (!v.is_table()) + return std::unexpected(std::string( + "expected a string or a { = \"...\" } table")); + static constexpr std::string_view kKnown[] = { + "linux", "macos", "macosx", "windows", "default", + }; + std::optional chosen, fallback; + for (auto& [k, val] : v.as_table()) { + if (std::ranges::find(kKnown, k) == std::ranges::end(kKnown)) + return std::unexpected(std::format( + "unknown platform key '{}'; expected one of linux, macos, windows, default", k)); + if (!val.is_string()) + return std::unexpected(std::format("platform key '{}' must be a string", k)); + const std::string_view canon = (k == "macosx") ? "macos" : std::string_view(k); + if (canon == host) chosen = val.as_string(); + else if (canon == "default") fallback = val.as_string(); + } + if (chosen) return chosen; + return fallback; // may be nullopt: not declared on this host +} + std::expected parse_string(std::string_view content, const std::filesystem::path& origin = "mcpp.toml", LoadContext ctx = {}); @@ -270,13 +321,19 @@ std::expected parse_string(std::string_view content, "target.*.build.flags", // #258 — middle segment is the cfg predicate "runtime.requirements", "runtime.artifacts", + // #544: `deps = [{ linux = "..." }]` — every entry a per-platform + // table — is the same Value shape as `[[xlings.deps]]`, and the guard + // cannot tell the inline form from the doubled-bracket typo. The + // reader below type-checks every entry, so nothing is silently + // dropped on this path either way. + "xlings.deps", }; if (auto badPath = find_disallowed_array_of_tables(doc->root(), "", kAllowedArraysOfTables)) { return std::unexpected(error(origin, std::format( "[[{}]] (array-of-tables) is not allowed for section '{}'; " "array-of-tables syntax is only supported for [[build.flags]], " - "[[features..flags]], [[runtime.requirements]], and " - "[[runtime.artifacts]]", + "[[features..flags]], [[runtime.requirements]], " + "[[runtime.artifacts]], and [xlings] deps entries", *badPath, *badPath))); } @@ -1347,15 +1404,30 @@ std::expected parse_string(std::string_view content, m.buildConfig.dependencyLinkage = *v; } - // [xlings] — build environment (L-1). Subsections mirror .xlings.json 1:1. - if (auto v = doc->get_string_array("xlings.deps")) m.xlings.deps = *v; + // [xlings] — build environment (L-1). Subsections mirror .xlings.json 1:1, + // including its per-platform value form, resolved here for this host + // (see resolve_host_value above). + if (auto* arr = doc->get("xlings.deps"); arr && arr->is_array()) { + std::size_t i = 0; + for (auto& el : arr->as_array()) { + auto r = resolve_host_value(el, host_platform_key()); + if (!r) return std::unexpected(error(origin, + std::format("[xlings] deps[{}]: {}", i, r.error()))); + if (*r) m.xlings.deps.push_back(**r); + ++i; + } + } if (doc->get("xlings.subos")) { m.xlings.subosDeclared = true; if (auto v = doc->get_string("xlings.subos")) m.xlings.subos = *v; } if (auto* wt = doc->get_table("xlings.workspace")) - for (auto& [k, val] : *wt) - if (val.is_string()) m.xlings.workspace[k] = val.as_string(); + for (auto& [k, val] : *wt) { + auto r = resolve_host_value(val, host_platform_key()); + if (!r) return std::unexpected(error(origin, + std::format("[xlings.workspace] {}: {}", k, r.error()))); + if (*r) m.xlings.workspace[k] = **r; + } if (auto* et = doc->get_table("xlings.envs")) for (auto& [k, val] : *et) if (val.is_string()) m.xlings.envs[k] = val.as_string(); @@ -1913,12 +1985,12 @@ std::expected parse_string(std::string_view content, } } - // Unsupported SCALAR keys are REPORTED, not dropped. + // Unsupported scalar and array keys are REPORTED, not dropped. // `[targets.]` has done this since #249; this table did not, // so a key that looks plausible — `cxx_runtime_tests` was the real // one — was accepted in silence and had no effect (#418). // - // ⚠️ SCALARS ONLY, AND THAT IS THE POINT. The sub-TABLES here are the + // ⚠️ NO SUB-TABLES, AND THAT IS THE POINT. The sub-TABLES here are the // conditional channel (`[target..build]`, `.dependencies`, // `.dev-dependencies`, `.build-dependencies`, `.feature-deps`) and // TOML presents each as a key of this table. A hand-written list of @@ -1930,32 +2002,33 @@ std::expected parse_string(std::string_view content, // warned about `[target.'cfg(unix)'.dependencies]`, a documented // feature with its own e2e. // - // Restricting the check to scalars removes the coupling entirely: + // Restricting the check to non-tables removes the coupling entirely: // new conditional sections need no change here, and the reported - // case — a scalar that does nothing — is still caught. + // case — a key that does nothing — is still caught. + // + // Scalars AND arrays (#544). The sweep used to skip arrays, which + // kept it from reporting `runner` as unsupported while honouring + // it — but it also let `runnerX = [...]` pass in silence, and its + // list of supported keys omitted the one array this table reads. + // Two lists, one per type, so an array typo is reported and a + // correctly spelled array is not; the message prints both in one + // alphabetical line, because a reader of the warning should not + // have to know a key's type to find it there. static constexpr std::string_view kKnownTargetScalars[] = { "cxx_runtime", "linkage", "sysroot", "toolchain", }; + static constexpr std::string_view kKnownTargetArrays[] = { "runner" }; for (auto& [key, value] : body) { if (value.is_table()) continue; // the conditional channel - // ...and arrays, which this sweep was never about. It checks - // SCALARS ("a scalar that does nothing"), and `runner` is an - // array read a few lines above — reaching here it was reported - // as "unsupported key 'runner' (ignored)" while in fact being - // honoured, which is worse than either being true. - if (value.is_array()) continue; - bool known = false; - for (auto k : kKnownTargetScalars) if (key == k) { known = true; break; } - if (known) continue; - std::string supported; - for (auto k : kKnownTargetScalars) { - if (!supported.empty()) supported += ", "; - supported += k; - } + const std::span known = value.is_array() + ? std::span(kKnownTargetArrays) + : std::span(kKnownTargetScalars); + if (std::ranges::find(known, key) != known.end()) continue; m.schemaWarnings.push_back(std::format( - "[target.{}] has unsupported key '{}' (ignored). Supported keys: {}. " + "[target.{}] has unsupported key '{}' (ignored). Supported keys: " + "cxx_runtime, linkage, runner, sysroot, toolchain. " "Per-role contracts go in [build].cxx_runtime's table form.", - triple, key, supported)); + triple, key)); } m.targetOverrides[canon_triple(triple)] = std::move(e); diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index 5ce20798..86b16e54 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -760,6 +760,11 @@ struct RuntimeConfig { // (env vars applied by xvm shims). See // .agents/docs/2026-06-29-manifest-environment-and-platform-design.md (L-1). struct XlingsConfig { + // Both lists are already resolved for THIS host: a manifest may write an + // entry or a value as `{ linux = "...", default = "..." }` (the form + // xlings' own `.xlings.json` accepts), and the parser keeps what applies + // here and drops what does not. Readers see a flat list and need no + // platform logic of their own. See `resolve_host_value` in toml.cppm. std::vector deps; // → .xlings.json "deps" std::map workspace; // → "workspace" (tool → version) std::string subos; // → "subos" (named project sandbox) diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index 62ac2e63..6e4f9a4c 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -2,6 +2,7 @@ import std; import mcpp.manifest; +import mcpp.libs.toml; import mcpp.pm.dep_spec; import mcpp.platform.axis; import mcpp.platform; @@ -4444,3 +4445,110 @@ during_build = { cmd = "play bgm.mp3", lopo = true } EXPECT_TRUE(found); EXPECT_TRUE(m->hooks.active()); } + +// ─── #544: array keys in the [target.] sweep; [xlings] per host ──── + +// The sweep skipped arrays by design (so that `runner`, an array it reads, was +// not reported as unsupported), which also let an array-valued typo pass in +// silence and left `runner` out of the "supported keys" list. Both halves of +// the issue's third ask are pinned here; TargetRunnerParsesAndDoesNotWarn +// above keeps the other direction (the real key stays unreported). +TEST(Manifest, TargetSweepReportsArrayTyposAndListsRunner) { + auto m = mcpp::manifest::parse_string(R"( +[package] +name = "app" +version = "0.1.0" +[target.aarch64-linux-musl] +runnerX = ["qemu-aarch64-static"] +)"); + ASSERT_TRUE(m.has_value()) << m.error().format(); + ASSERT_EQ(m->schemaWarnings.size(), 1u); + EXPECT_NE(m->schemaWarnings[0].find("'runnerX'"), std::string::npos) << m->schemaWarnings[0]; + EXPECT_NE(m->schemaWarnings[0].find( + "Supported keys: cxx_runtime, linkage, runner, sysroot, toolchain"), + std::string::npos) << m->schemaWarnings[0]; +} + +// `[xlings]` mirrors `.xlings.json`, whose values may be an object keyed by +// platform. The parser resolves the form against this host at load, so the +// assertions below are written in terms of host_platform_key() rather than a +// fixed platform: the same test file runs on all three CI hosts. +TEST(Manifest, XlingsDepsAcceptPerPlatformEntries) { + auto m = mcpp::manifest::parse_string(R"( +[package] +name = "app" +version = "0.1.0" +[xlings] +deps = ["xim:ninja", { linux = "qemu-user-aarch64" }, { windows = "nasm", default = "yasm" }] +)"); + ASSERT_TRUE(m.has_value()) << m.error().format(); + const auto host = mcpp::manifest::host_platform_key(); + std::vector want{"xim:ninja"}; + if (host == "linux") want.push_back("qemu-user-aarch64"); + want.push_back(host == "windows" ? "nasm" : "yasm"); + EXPECT_EQ(m->xlings.deps, want); +} + +TEST(Manifest, XlingsWorkspaceAcceptsPerPlatformValues) { + auto m = mcpp::manifest::parse_string(R"( +[package] +name = "app" +version = "0.1.0" +[xlings.workspace] +gcc = { linux = "15.1.0" } +llvm = { macos = "20", default = "22" } +xmake = "3.0.7" +)"); + ASSERT_TRUE(m.has_value()) << m.error().format(); + const auto host = mcpp::manifest::host_platform_key(); + if (host == "linux") EXPECT_EQ(m->xlings.workspace.at("gcc"), "15.1.0"); + else EXPECT_EQ(m->xlings.workspace.count("gcc"), 0u); + EXPECT_EQ(m->xlings.workspace.at("llvm"), host == "macos" ? "20" : "22"); + EXPECT_EQ(m->xlings.workspace.at("xmake"), "3.0.7"); +} + +TEST(Manifest, XlingsUnknownPlatformKeyIsAnError) { + auto m = mcpp::manifest::parse_string(R"( +[package] +name = "app" +version = "0.1.0" +[xlings] +deps = [{ linxu = "qemu-user-aarch64" }] +)"); + ASSERT_FALSE(m.has_value()); + EXPECT_NE(m.error().message.find("linxu"), std::string::npos) << m.error().message; + EXPECT_NE(m.error().message.find("linux, macos, windows, default"), std::string::npos) + << m.error().message; + EXPECT_NE(m.error().message.find("deps[0]"), std::string::npos) << m.error().message; +} + +TEST(Manifest, XlingsWorkspaceNonStringLeafIsAnError) { + auto m = mcpp::manifest::parse_string(R"( +[package] +name = "app" +version = "0.1.0" +[xlings.workspace] +gcc = { linux = 15 } +)"); + ASSERT_FALSE(m.has_value()); + EXPECT_NE(m.error().message.find("[xlings.workspace] gcc"), std::string::npos) + << m.error().message; +} + +// The resolution rule itself, with the platform passed explicitly so all +// three hosts' answers are asserted on every host. +TEST(Manifest, ResolveHostValueTable) { + using mcpp::manifest::resolve_host_value; + auto d = mcpp::libs::toml::parse(R"(v = { linux = "a", macosx = "b", default = "c" })"); + ASSERT_TRUE(d.has_value()); + const auto& v = d->root().at("v"); + EXPECT_EQ(resolve_host_value(v, "linux").value().value(), "a"); + EXPECT_EQ(resolve_host_value(v, "macos").value().value(), "b"); // macosx alias + EXPECT_EQ(resolve_host_value(v, "windows").value().value(), "c"); + auto e = mcpp::libs::toml::parse(R"(v = { linux = "a" })"); + ASSERT_TRUE(e.has_value()); + EXPECT_FALSE(resolve_host_value(e->root().at("v"), "windows").value().has_value()); + auto s = mcpp::libs::toml::parse(R"(v = "plain")"); + ASSERT_TRUE(s.has_value()); + EXPECT_EQ(resolve_host_value(s->root().at("v"), "windows").value().value(), "plain"); +} From 7501e50af6d0892192fe1d3689cda67b082f85df Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 2 Sep 2026 11:47:02 +0800 Subject: [PATCH 03/12] build: runner lookup through declared payloads then PATH, with typed messages (#544) --- src/build/runner_lookup.cppm | 138 ++++++++++++++++++++++++++++++ tests/unit/test_runner_lookup.cpp | 128 +++++++++++++++++++++++++++ 2 files changed, 266 insertions(+) create mode 100644 src/build/runner_lookup.cppm create mode 100644 tests/unit/test_runner_lookup.cpp diff --git a/src/build/runner_lookup.cppm b/src/build/runner_lookup.cppm new file mode 100644 index 00000000..5b86b293 --- /dev/null +++ b/src/build/runner_lookup.cppm @@ -0,0 +1,138 @@ +// mcpp.build.runner_lookup — where the runner's program is, and what to say +// when it is not. +// +// The lookup is mcpp's own rather than posix_spawnp's for one measured reason: +// a bare name on PATH resolves to an xvm shim, and the shim answers for the +// current subos rather than for the package (e2e 130 in CI: `mcpp run` exec'ing +// the bare `qemu-system-riscv64` answered "not installed" while +// `qemu-system-riscv64 --version` in the same job succeeded; `python3` on the +// development host answers "not installed in this subos (_) — installed +// elsewhere"). The payload's bin/ is the binary itself, so it is searched +// first. That is what lets a runner name a program the project declared under +// `[xlings] deps` without writing the payload's home-and-version path into the +// manifest — the thing BuildConfig::runner's comment says a static manifest +// cannot do. +// +// Doing the lookup here has a second effect: "not found anywhere" is decided +// before any spawn, so a spawn-time ENOENT can only mean the program was found +// and its interpreter or loader was not. The two messages differ, and neither +// guesses. +// +// Design: .agents/docs/2026-09-02-runner-beyond-baremetal-design.md §4.3-4.4. + +module; +#include + +export module mcpp.build.runner_lookup; + +import std; +import mcpp.platform; + +export namespace mcpp::build::runner_lookup { + +struct Lookup { + std::optional program; // absolute, executable + std::vector searched; // in order, for the message +}; + +namespace detail { +inline bool executable_file(const std::filesystem::path& p) { + std::error_code ec; + if (!std::filesystem::is_regular_file(p, ec)) return false; + if constexpr (mcpp::platform::is_windows) return true; + auto perms = std::filesystem::status(p, ec).permissions(); + using P = std::filesystem::perms; + return (perms & (P::owner_exec | P::group_exec | P::others_exec)) != P::none; +} +} // namespace detail + +// `argv0` absolute, or containing a directory separator: taken as-is when it +// is an executable file. Otherwise `/argv0`, then each `PATH` +// entry (`pathEnv` split on the platform's list separator); the first +// executable regular file wins. Every directory looked in is recorded so the +// not-found message can list them. +inline Lookup locate(std::string_view argv0, + std::span depBinDirs, + std::string_view pathEnv) +{ + Lookup out; + std::filesystem::path a0(argv0); + const bool hasDir = a0.is_absolute() + || argv0.find('/') != std::string_view::npos + || argv0.find('\\') != std::string_view::npos; + if (hasDir) { + std::error_code ec; + if (detail::executable_file(a0)) out.program = std::filesystem::absolute(a0, ec); + out.searched.push_back(a0.parent_path()); + return out; + } + for (auto const& d : depBinDirs) { + out.searched.push_back(d); + if (auto c = d / a0; detail::executable_file(c)) { out.program = c; return out; } + } + constexpr char sep = mcpp::platform::is_windows ? ';' : ':'; + for (auto part : std::views::split(pathEnv, sep)) { + std::string_view sv(part.begin(), part.end()); + if (sv.empty()) continue; + std::filesystem::path d(sv); + out.searched.push_back(d); + if (auto c = d / a0; detail::executable_file(c)) { out.program = c; return out; } + } + return out; +} + +// What the kernel's refusal means. Only ENOEXEC (and EBADARCH where the +// platform defines it) says "this host cannot load the artifact"; everything +// else — EACCES, ENOENT on a found program, E2BIG — is reported verbatim and +// never turned into advice about runners. +enum class SpawnClass { Unloadable, Other }; + +inline SpawnClass classify(int e) { + if (e == ENOEXEC) return SpawnClass::Unloadable; +#if defined(EBADARCH) + if (e == EBADARCH) return SpawnClass::Unloadable; +#endif + return SpawnClass::Other; +} + +inline std::string errno_text(int e) { + return std::generic_category().message(e); +} + +inline std::string not_found_message(std::string_view triple, std::string_view argv0, + std::span searched) { + std::string dirs; + for (auto const& d : searched) dirs += "\n " + d.string(); + return std::format( + "runner '{}' for '{}' was not found. Searched:{}\n" + " Declare the package that provides it under [xlings] deps, or " + "install it on PATH.\n" + " Pass --no-runner to execute the artifact directly on this host.", + argv0, triple, dirs); +} + +inline std::string spawn_failed_message(std::string_view program, int e) { + return std::format("'{}' could not be started: {} (error {})", + program, errno_text(e), e); +} + +// The hosted sibling of mcpp::freestanding::no_runner_message. It reports what +// the kernel answered rather than asserting why: on a hosted triple mcpp does +// not know whether the refusal is a foreign ISA or a file that is not an +// executable at all, and the example it prints is a user-mode emulator. +inline std::string unrunnable_message(std::string_view triple, + const std::filesystem::path& artifact, int e) { + return std::format( + "this host cannot execute '{}': {} (error {}).\n" + " The artifact was built for '{}'. Declare how to run it here:\n" + "\n" + " [target.{}]\n" + " runner = [\"qemu-aarch64-static\"]\n" + "\n" + " The artifact path is appended, or substituted for `{{}}` if the " + "template contains it.\n" + " A host that can execute it directly may pass --no-runner.", + artifact.string(), errno_text(e), e, triple, triple); +} + +} // namespace mcpp::build::runner_lookup diff --git a/tests/unit/test_runner_lookup.cpp b/tests/unit/test_runner_lookup.cpp new file mode 100644 index 00000000..279ff3de --- /dev/null +++ b/tests/unit/test_runner_lookup.cpp @@ -0,0 +1,128 @@ +#include +#include +#include + +import std; +import mcpp.build.runner_lookup; + +using namespace mcpp::build::runner_lookup; + +// The lookup order is the whole point (#544 §4.4): a declared payload's bin/ +// beats PATH, because a bare name on PATH reaches an xvm shim that answers for +// the current subos rather than for the package. The directories are real and +// the files are executable, so what is asserted is the rule, not a mock of it. +#if !defined(_WIN32) + +namespace { +std::filesystem::path fresh_root(std::string_view name) { + auto root = std::filesystem::temp_directory_path() / name; + std::filesystem::remove_all(root); + std::filesystem::create_directories(root); + return root; +} +std::filesystem::path make_exe(const std::filesystem::path& dir, std::string_view name) { + std::filesystem::create_directories(dir); + auto p = dir / name; + { std::ofstream o(p); o << "#!/bin/sh\nexit 0\n"; } + std::filesystem::permissions(p, std::filesystem::perms::owner_all); + return p; +} +} // namespace + +TEST(RunnerLookup, PayloadBinBeatsPath) { + auto root = fresh_root("mcpp-runner-lookup-1"); + auto inPayload = make_exe(root / "payload" / "bin", "qemu-x"); + make_exe(root / "path", "qemu-x"); + std::vector bins{root / "payload" / "bin"}; + auto l = locate("qemu-x", bins, (root / "path").string()); + ASSERT_TRUE(l.program.has_value()); + EXPECT_EQ(*l.program, inPayload); +} + +TEST(RunnerLookup, PathIsSearchedAfterPayloads) { + auto root = fresh_root("mcpp-runner-lookup-2"); + auto onPath = make_exe(root / "path", "qemu-y"); + std::vector bins{root / "payload" / "bin"}; // absent dir + auto l = locate("qemu-y", bins, (root / "path").string()); + ASSERT_TRUE(l.program.has_value()); + EXPECT_EQ(*l.program, onPath); + ASSERT_EQ(l.searched.size(), 2u); + EXPECT_EQ(l.searched[0], root / "payload" / "bin"); + EXPECT_EQ(l.searched[1], root / "path"); +} + +TEST(RunnerLookup, NonExecutableFileIsSkipped) { + auto root = fresh_root("mcpp-runner-lookup-3"); + std::filesystem::create_directories(root / "p1"); + { std::ofstream o(root / "p1" / "tool"); o << "data"; } // no exec bit + auto real = make_exe(root / "p2", "tool"); + auto l = locate("tool", {}, (root / "p1").string() + ":" + (root / "p2").string()); + ASSERT_TRUE(l.program.has_value()); + EXPECT_EQ(*l.program, real); +} + +TEST(RunnerLookup, NotFoundListsEveryDirectorySearched) { + auto root = fresh_root("mcpp-runner-lookup-4"); + std::vector bins{root / "a" / "bin"}; + auto l = locate("nope", bins, (root / "p1").string() + ":" + (root / "p2").string()); + EXPECT_FALSE(l.program.has_value()); + ASSERT_EQ(l.searched.size(), 3u); + auto msg = not_found_message("aarch64-linux-musl", "nope", l.searched); + EXPECT_NE(msg.find("runner 'nope'"), std::string::npos) << msg; + EXPECT_NE(msg.find("aarch64-linux-musl"), std::string::npos) << msg; + EXPECT_NE(msg.find((root / "a" / "bin").string()), std::string::npos) << msg; + EXPECT_NE(msg.find((root / "p2").string()), std::string::npos) << msg; + EXPECT_NE(msg.find("[xlings] deps"), std::string::npos) << msg; + EXPECT_NE(msg.find("--no-runner"), std::string::npos) << msg; +} + +TEST(RunnerLookup, AbsoluteArgv0IsTakenAsIs) { + auto root = fresh_root("mcpp-runner-lookup-5"); + auto abs = make_exe(root, "runner.sh"); + auto l = locate(abs.string(), {}, ""); + ASSERT_TRUE(l.program.has_value()); + EXPECT_EQ(*l.program, abs); + // ...and an absolute path that is not there is not searched for elsewhere. + auto missing = locate((root / "absent.sh").string(), {}, root.string()); + EXPECT_FALSE(missing.program.has_value()); +} + +TEST(RunnerLookup, EmptyPathEntriesAreIgnored) { + auto root = fresh_root("mcpp-runner-lookup-6"); + auto onPath = make_exe(root / "p", "tool"); + auto l = locate("tool", {}, ":" + (root / "p").string() + "::"); + ASSERT_TRUE(l.program.has_value()); + EXPECT_EQ(*l.program, onPath); + EXPECT_EQ(l.searched.size(), 1u); +} + +TEST(RunnerLookup, ClassifiesENOEXECAsUnloadable) { + EXPECT_EQ(classify(ENOEXEC), SpawnClass::Unloadable); + EXPECT_EQ(classify(EACCES), SpawnClass::Other); + EXPECT_EQ(classify(ENOENT), SpawnClass::Other); + EXPECT_EQ(classify(0), SpawnClass::Other); +} + +TEST(RunnerLookup, UnrunnableMessageNamesKernelAnswerTripleAndKey) { + auto msg = unrunnable_message("aarch64-linux-musl", "/x/bin/app", ENOEXEC); + EXPECT_NE(msg.find("Exec format error"), std::string::npos) << msg; + EXPECT_NE(msg.find("/x/bin/app"), std::string::npos) << msg; + EXPECT_NE(msg.find("built for 'aarch64-linux-musl'"), std::string::npos) << msg; + EXPECT_NE(msg.find("[target.aarch64-linux-musl]"), std::string::npos) << msg; + EXPECT_NE(msg.find("runner = [\"qemu-aarch64-static\"]"), std::string::npos) << msg; + EXPECT_NE(msg.find("--no-runner"), std::string::npos) << msg; +} + +TEST(RunnerLookup, SpawnFailedMessageIsVerbatim) { + auto msg = spawn_failed_message("/x/bin/qemu", EACCES); + EXPECT_NE(msg.find("'/x/bin/qemu' could not be started"), std::string::npos) << msg; + EXPECT_NE(msg.find("Permission denied"), std::string::npos) << msg; + EXPECT_NE(msg.find("(error 13)"), std::string::npos) << msg; + EXPECT_EQ(msg.find("runner = ["), std::string::npos) << msg; // no runner advice +} + +#else + +TEST(RunnerLookup, WindowsCoveredByIntegration) { SUCCEED(); } + +#endif From be0beee2f7ba1f2dee0ee49f8d5e0596c2156b0b Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 2 Sep 2026 12:00:23 +0800 Subject: [PATCH 04/12] run/test: honour the declared runner on every target; --no-runner; not-run reporting with exit 2; e2e 330 (#544) --- src/build/execute.cppm | 385 +++++++++++++++++++------ src/build/prepare.cppm | 23 ++ src/build/runner_lookup.cppm | 4 +- src/cli.cppm | 6 +- src/cli/cmd_build.cppm | 54 +++- tests/e2e/330_runner_hosted_targets.sh | 160 ++++++++++ 6 files changed, 534 insertions(+), 98 deletions(-) create mode 100644 tests/e2e/330_runner_hosted_targets.sh diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 0bf3eaf4..ebbf0738 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -16,6 +16,7 @@ import mcpp.diag; import mcpp.build.plan; import mcpp.toolchain.triple; import mcpp.freestanding.runner; +import mcpp.build.runner_lookup; // #544: where the runner's program is import mcpp.freestanding.linkline; import mcpp.build.graph_shape; // #407: which mode wrote this build.ninja import mcpp.build.backend; @@ -115,6 +116,14 @@ struct BuildCacheEntry { // list gets written. Same discipline as `subosRecorded` above, and for the // same reason. bool depSourceRootsRecorded = false; + // Did the build this entry records have a runner declared for its target + // (#544)? The run fast path executes the artifact bare and has no manifest + // to read a template from, so an entry with a runner is a miss for it — + // the prepare path then consults choose_runner as the first `mcpp run` + // did. Absent on caches written before the field: false, which is the + // pre-#544 behaviour and correct for every entry such a cache could hold + // (a hosted runner was never consulted then, so none was ever used). + bool runnerDeclared = false; }; std::vector read_build_cache(const std::filesystem::path& projectRoot) { @@ -220,6 +229,11 @@ std::vector read_build_cache(const std::filesystem::path& proje e.depSourceRootsRecorded = true; haveNextLine = static_cast(std::getline(f, line)); } + // Optional `runner=0|1` (#544). Absent ⇒ false; see the field. + if (haveNextLine && line.starts_with("runner=")) { + e.runnerDeclared = (line.substr(7) == "1"); + haveNextLine = static_cast(std::getline(f, line)); + } entries.push_back(std::move(e)); if (!haveNextLine || line.empty()) break; } @@ -244,7 +258,8 @@ void write_build_cache(const std::filesystem::path& projectRoot, const std::string& profile = "", const std::string& cacheMode = "", const mcpp::platform::runtime::RuntimeBinding& runtimeBinding = {}, - std::vector depSourceRoots = {}) { + std::vector depSourceRoots = {}, + bool runnerDeclared = false) { auto path = projectRoot / kBuildCacheFile; auto entries = read_build_cache(projectRoot); @@ -265,6 +280,7 @@ void write_build_cache(const std::filesystem::path& projectRoot, newEntry.runtimeBinding = runtimeBinding; newEntry.depSourceRoots = std::move(depSourceRoots); newEntry.depSourceRootsRecorded = true; + newEntry.runnerDeclared = runnerDeclared; entries.insert(entries.begin(), std::move(newEntry)); // Trim to LRU capacity. @@ -305,6 +321,7 @@ void write_build_cache_entries(const std::filesystem::path& path, f << "cacheMode=" << e.cacheMode << '\n'; f << "depSourceRoots=" << e.depSourceRoots.size() << '\n'; for (auto& r : e.depSourceRoots) f << r << '\n'; + f << "runner=" << (e.runnerDeclared ? 1 : 0) << '\n'; } } @@ -424,30 +441,47 @@ compute_subos_env(const mcpp::build::BuildPlan& plan) { // does not fail when you add the second derivation, it fails later, when one // of them gains a rule the other does not. // -// Returns an empty argv for a hosted target — the caller runs the artifact -// directly, as it always did. +// Returns an empty argv when nothing is declared — the caller runs the +// artifact directly. +// +// Read for EVERY target (#544). The freestanding predicate used to gate this +// read, which is how a runner declared under a hosted cross triple +// (`[target.aarch64-linux-musl].runner` on an x86_64 host) was parsed, +// validated, documented and never consulted: `mcpp run` exec'd the artifact +// bare, the kernel refused it with ENOEXEC, and nothing said so. The +// predicate now decides exactly one thing — whether an absent runner is fatal +// before any spawn is attempted — and `RunnerChoice::freestanding` keeps that +// meaning. Whether THIS host can execute a hosted artifact is not predicted +// here or anywhere: mcpp does what the project declared, or attempts the +// launch and reports what the kernel answered (design §3, P1). struct RunnerChoice { std::vector tmpl; // empty = execute the artifact directly - bool freestanding = false; // a runner is REQUIRED when true + bool freestanding = false; // an EMPTY tmpl is fatal when true bool fromManifest = false; // the consumer overrode a dependency's + bool ignored = false; // --no-runner dropped a declared template }; -RunnerChoice choose_runner(const BuildContext& ctx) { +RunnerChoice choose_runner(const BuildContext& ctx, bool noRunner = false) { RunnerChoice c; - auto ft = mcpp::toolchain::triple::parse(ctx.tc.targetTriple); - if (!ft || !ft->is_freestanding()) return c; - c.freestanding = true; + if (auto ft = mcpp::toolchain::triple::parse(ctx.tc.targetTriple)) + c.freestanding = ft->is_freestanding(); // Two producers, ordinary precedence: what the author of THIS project // wrote beats what a dependency supplied. The dependency is the normal - // case (a board-support package computes the emulator's absolute path); - // the manifest key exists for swapping `-bios default` for - // `-bios none -semihosting` while debugging. + // case on bare metal (a board-support package computes the emulator's + // absolute path); the manifest key exists for swapping `-bios default` + // for `-bios none -semihosting` while debugging, and — on a hosted cross + // triple — for naming the user-mode emulator at all. c.tmpl = ctx.manifest.buildConfig.runner; if (auto it = ctx.manifest.targetOverrides.find(ctx.tc.targetTriple); it != ctx.manifest.targetOverrides.end() && !it->second.runner.empty()) { c.tmpl = it->second.runner; c.fromManifest = !ctx.manifest.buildConfig.runner.empty(); } + // `--no-runner` is the operator on THIS host stating a host fact the + // manifest cannot carry: the triple is native here. On a freestanding + // target that leaves nothing to execute, and the caller's existing + // no-runner error is the correct answer. + if (noRunner && !c.tmpl.empty()) { c.tmpl.clear(); c.ignored = true; } return c; } @@ -643,7 +677,10 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, for (auto const& r : ctx.depSourceRoots) v.push_back(r.generic_string()); return v; - }()); + }(), + // #544: the run fast path declines an entry whose + // target has a runner declared — see the field. + !choose_runner(ctx).tmpl.empty()); } // The one place the --strict policy is settled. Degradations reported by @@ -1070,6 +1107,13 @@ std::optional try_fast_run(const std::filesystem::path& projectRoot, } } if (!match || match->runTargets.empty()) return std::nullopt; + // A runner declared for the host target (a wrapper such as valgrind, or + // a triple that is native here but carries an emulator) is consulted on + // the prepare path through choose_runner. This path has no manifest to + // read the template from, and executing the artifact bare here while the + // other door wraps it would make the second `mcpp run` behave differently + // from the first. The entry records the fact; the fast path declines. + if (match->runnerDeclared) return std::nullopt; auto outputDirStr = match->outputDir; auto ninjaProgram = match->ninjaProgram; @@ -1178,7 +1222,23 @@ std::optional try_fast_run(const std::filesystem::path& projectRoot, })) childEnv.push_back(std::move(kv)); - return mcpp::platform::process::run_exec(argv, childEnv) == 0 ? 0 : 1; + // Same contract as the prepare path below: a refused spawn is reported + // and exits 2, never folded into the artifact's own exit status (#544). + // No runner can be declared for an entry this path accepts (see the + // `runnerDeclared` gate above), so the artifact is the only thing that + // could have been refused. + int spawnErr = 0; + const int exitRc = mcpp::platform::process::run_exec(argv, childEnv, &spawnErr); + if (spawnErr != 0) { + using namespace mcpp::build::runner_lookup; + const auto triple = mcpp::toolchain::triple::host_triple().str(); + if (classify(spawnErr) == SpawnClass::Unloadable) + std::println(stderr, "error: {}", unrunnable_message(triple, exe, spawnErr)); + else + std::println(stderr, "error: {}", spawn_failed_message(exe.string(), spawnErr)); + return 2; + } + return exitRc == 0 ? 0 : 1; } // `mcpp run` driver: build, locate the binary target, exec it with the @@ -1193,7 +1253,8 @@ export int build_run_target(const std::optional& targetName, const std::string& package_filter = {}, const std::string& cache_mode = {}, bool no_cache = false, - const std::string& target_triple = {}) { + const std::string& target_triple = {}, + bool no_runner = false) { // mcpp#225 (E2): reuse the resolved build cache when it's still fresh, // skipping prepare_build's toolchain resolution + modgraph scan // entirely — mirrors cmd_build's try_fast_build fast path. The cached @@ -1204,8 +1265,12 @@ export int build_run_target(const std::optional& targetName, // A --cache/--no-cache override also bypasses the fast path, for the same // reason --profile does: the cached build.ninja was generated under the // previous mode, so reusing it would silently ignore the flag. + // `--no-runner` bypasses it too: the fast path executes the artifact bare, + // and it only takes an entry that records no runner (see runnerDeclared), + // so with the flag there is nothing for it to ignore — and it has no + // manifest to print the note against. if (package_filter.empty() && cache_mode.empty() && !no_cache - && target_triple.empty()) { + && target_triple.empty() && !no_runner) { if (auto root = mcpp::project::find_manifest_root(std::filesystem::current_path())) { if (auto rc = try_fast_run(*root, targetName, passthrough)) { return *rc; @@ -1261,41 +1326,57 @@ export int build_run_target(const std::optional& targetName, auto exe = ctx->outputDir / chosen->output; auto pathCtx = mcpp::fetcher::make_path_ctx(/*cfg=*/nullptr, ctx->projectRoot); std::vector argv; - // A freestanding artifact is not executable on this machine by - // construction — wrong ISA, no loader, and it expects to own the address - // space. Exec'ing it directly gives "Exec format error", which describes - // the symptom and not the situation. The runner template says how to stand - // something in front of it; mcpp never guesses one, because which emulator - // and which machine model are board facts (see mcpp.freestanding.runner). - bool freestandingRun = false; - if (auto ft = mcpp::toolchain::triple::parse(ctx->tc.targetTriple)) - freestandingRun = ft->is_freestanding(); - if (freestandingRun) { - // Two producers, and the precedence is the ordinary one: what the - // author of THIS project wrote beats what a dependency supplied. - // - // The dependency is the normal case — a board-support package knows - // the emulator, its machine model and its firmware mode, and computes - // the absolute path that a static manifest cannot. The explicit key - // exists for the other case: swapping `-bios default` for - // `-bios none -semihosting` while debugging is a legitimate thing to - // want, and removing that ability to make the BSP authoritative would - // trade one problem for a worse one. - const auto choice = choose_runner(*ctx); + // An artifact this machine cannot execute — a freestanding image by + // construction, a hosted cross artifact by circumstance — needs something + // to stand in front of it. The runner template says what; mcpp never + // guesses one, because which emulator and which machine model are facts + // about the board or the host (see mcpp.freestanding.runner and + // mcpp.build.runner_lookup). One read point decides for both `run` and + // `test`: choose_runner. + // + // Two producers, and the precedence is the ordinary one: what the author + // of THIS project wrote beats what a dependency supplied. The dependency + // is the normal case on bare metal — a board-support package knows the + // emulator, its machine model and its firmware mode, and computes the + // absolute path that a static manifest cannot. The explicit key exists for + // the other case: swapping `-bios default` for `-bios none -semihosting` + // while debugging, or naming `qemu-aarch64-static` for a cross target. + const auto choice = choose_runner(*ctx, no_runner); + if (choice.ignored) + mcpp::ui::info("note", std::format( + "--no-runner: ignoring the runner declared for {}", ctx->tc.targetTriple)); + if (choice.fromManifest) + mcpp::ui::info("note", std::format( + "[target.{}].runner overrides the runner a dependency supplied", + ctx->tc.targetTriple)); + if (choice.freestanding && choice.tmpl.empty()) { + std::println(stderr, "error: {}", + mcpp::freestanding::no_runner_message(ctx->tc.targetTriple)); + return 2; + } + if (!choice.tmpl.empty()) { + // The program is located by mcpp, not by posix_spawnp: a declared + // payload's bin/ first, then PATH — see runner_lookup for the shim + // measurement that makes the order matter. Not found anywhere is + // decided here, before any spawn, and is an error rather than a + // fallback to bare execution (#544, D1): running the artifact under a + // different interpreter with different arguments is the failure the + // runner key exists to prevent. auto tmpl = choice.tmpl; - if (choice.fromManifest) - mcpp::ui::info("note", std::format( - "[target.{}].runner overrides the runner a dependency supplied", - ctx->tc.targetTriple)); - if (tmpl.empty()) { + const char* pathEnv = std::getenv("PATH"); + auto found = mcpp::build::runner_lookup::locate( + tmpl.front(), ctx->xlingsDepBinDirs, pathEnv ? pathEnv : ""); + if (!found.program) { std::println(stderr, "error: {}", - mcpp::freestanding::no_runner_message(ctx->tc.targetTriple)); + mcpp::build::runner_lookup::not_found_message( + ctx->tc.targetTriple, tmpl.front(), found.searched)); return 2; } + tmpl.front() = found.program->string(); argv = mcpp::freestanding::expand(tmpl, exe); for (auto& a : passthrough) argv.push_back(a); mcpp::ui::status("Running", std::format( - "`{} … {}`", tmpl.front(), + "`{} … {}`", choice.tmpl.front(), mcpp::ui::shorten_path(exe, pathCtx))); } else { argv.push_back(exe.string()); @@ -1316,7 +1397,28 @@ export int build_run_target(const std::optional& targetName, // Direct exec (no /bin/sh): the loader env reaches ONLY the target child, // never mcpp or a host shell. Fixes the bundled-glibc-vs-host-libtinfo // crash on newer-glibc distros. - return mcpp::platform::process::run_exec(argv, childEnv) == 0 ? 0 : 1; + // + // A refused spawn is typed and reported here, and exits 2 — the code the + // freestanding no-runner path already uses for "could not start", distinct + // from 1, which keeps meaning "ran and failed". With a runner the failure + // is the runner's own (verbatim errno, no advice); without one, ENOEXEC + // is the kernel saying this host cannot load the artifact, and the message + // carries the key that would change that. Anything else is reported as + // itself — EACCES is a permission problem, not an absence. + int spawnErr = 0; + const int rc = mcpp::platform::process::run_exec(argv, childEnv, &spawnErr); + if (spawnErr != 0) { + using namespace mcpp::build::runner_lookup; + if (!choice.tmpl.empty()) + std::println(stderr, "error: {}", spawn_failed_message(argv.front(), spawnErr)); + else if (classify(spawnErr) == SpawnClass::Unloadable) + std::println(stderr, "error: {}", + unrunnable_message(ctx->tc.targetTriple, exe, spawnErr)); + else + std::println(stderr, "error: {}", spawn_failed_message(exe.string(), spawnErr)); + return 2; + } + return rc == 0 ? 0 : 1; } export enum class TestMessageFormat { Human, Json }; @@ -1325,6 +1427,10 @@ export struct TestOptions { std::string filter; // substring match on the path-based test name; empty = all TestMessageFormat format = TestMessageFormat::Human; bool list = false; // enumerate only, no build/run + // `--no-runner`: run the test binaries directly, ignoring a declared + // runner — the operator on this host stating that the triple is native + // here, a fact the manifest has no axis for (#544, D3). + bool noRunner = false; // Per-test RUN deadline. The default is deliberately non-zero: `mcpp test` // is something CI runs unattended, and an unbounded default makes a single // hung test able to consume the whole job with nothing to show for it. @@ -1351,6 +1457,12 @@ export struct TestOptions { export struct TestRunSummary { int passed = 0; int failed = 0; + // Tests that were built and not executed, with the one reason that applies + // to all of them (#544): this host cannot load the artifacts, or the + // declared runner could not be found or started. Not a failure — the + // test did not run — and not a pass either: the exit code is 2. + int notRun = 0; + std::string notRunReason; long long buildMs = 0; // Phase A + bulk pass + per-test drives long long runMs = 0; // the test binaries' own execution long long elapsedMs = 0; // wall clock for the whole member @@ -1526,12 +1638,18 @@ export int run_tests(std::span passthrough, // the rest still build and run. struct TestResult { std::string name; - enum class St { Pass, CompileFail, RunFail } status; + // `NotRun` (#544): built, and not executed — the host cannot load the + // artifact, or the declared runner could not be found or started. + // Reporting that as `RunFail (exit 127)` states that the test ran and + // returned 127, which is false and indistinguishable from a missing + // program; reporting it as a pass would be read as one. + enum class St { Pass, CompileFail, RunFail, NotRun } status; int exitCode = 0; std::string compileOutput; std::string runOutput; long long durationMs = 0; // build+run wall time for THIS test bool timedOut = false; // killed by --timeout + std::string reason; // NotRun only: why, in one sentence }; std::vector results; @@ -1542,17 +1660,20 @@ export int run_tests(std::span passthrough, if (!json) return; const char* st = r.status == TestResult::St::Pass ? "pass" : r.status == TestResult::St::CompileFail ? "compile_fail" - : "run_fail"; + : r.status == TestResult::St::NotRun ? "not_run" + : "run_fail"; std::string signal = (r.exitCode > 128 && r.exitCode < 128 + 65) ? std::to_string(r.exitCode - 128) : "null"; std::println("{{\"member\":\"{}\",\"test\":\"{}\",\"status\":\"{}\"," "\"exit_code\":{},\"signal\":{}," "\"duration_ms\":{},\"timed_out\":{}," - "\"compile_output\":\"{}\",\"run_output\":\"{}\"}}", + "\"compile_output\":\"{}\",\"run_output\":\"{}\"," + "\"reason\":\"{}\"}}", test_json_escape(memberName), test_json_escape(r.name), st, r.exitCode, signal, r.durationMs, r.timedOut ? "true" : "false", - test_json_escape(r.compileOutput), test_json_escape(r.runOutput)); + test_json_escape(r.compileOutput), test_json_escape(r.runOutput), + test_json_escape(r.reason)); std::fflush(stdout); }; @@ -1708,6 +1829,48 @@ export int run_tests(std::span passthrough, // the terminal interleaves them line by line, which does not just look // untidy — it makes a failing assertion unattributable, and the whole // reason the per-test loop exists is attribution. + // How the test binaries are executed — the SAME runner `mcpp run` uses, + // resolved ONCE per invocation (#544). One read point, two callers; and + // one lookup, because every test of an invocation shares a target, so + // "the runner's program is not there" is a fact about the invocation and + // is reported once rather than once per test. + // + // Nothing else about the test model changes, and that is a measured + // result rather than a simplification: semihosting propagates the + // firmware's `main` return value to the emulator's exit code + // (`return 7` → qemu exits 7, verified), so "exit code is the verdict" + // holds under a runner exactly as it does on the host. + const auto runnerChoice = choose_runner(*ctx, testOpts.noRunner); + if (runnerChoice.ignored && !json) + mcpp::ui::info("note", std::format( + "--no-runner: ignoring the runner declared for {}", ctx->tc.targetTriple)); + if (runnerChoice.fromManifest && !json) + mcpp::ui::info("note", std::format( + "[target.{}].runner overrides the runner a dependency supplied", + ctx->tc.targetTriple)); + if (runnerChoice.freestanding && runnerChoice.tmpl.empty()) { + std::println(stderr, "error: {}", + mcpp::freestanding::no_runner_message(ctx->tc.targetTriple)); + return 2; + } + std::vector runnerTmpl = runnerChoice.tmpl; + // Non-empty ⇒ no test is spawned; every one is reported NotRun with it. + std::string invocationNotRunReason; + if (!runnerTmpl.empty()) { + const char* pathEnv = std::getenv("PATH"); + auto found = mcpp::build::runner_lookup::locate( + runnerTmpl.front(), ctx->xlingsDepBinDirs, pathEnv ? pathEnv : ""); + if (found.program) runnerTmpl.front() = found.program->string(); + else invocationNotRunReason = mcpp::build::runner_lookup::not_found_message( + ctx->tc.targetTriple, runnerTmpl.front(), found.searched); + } + // Set by the first worker whose spawn the kernel refused; every worker + // checks it before spawning. Workers already past the check may be + // refused the same way — harmless, a refused spawn has no side effects — + // and each such result is NotRun, not RunFail. The reason is printed once. + std::atomic hostCannotRun{false}; + std::string hostCannotRunReason; + auto run_tests_now = [&](std::vector& list) { if (list.empty()) return; const bool capture = json || list.size() > 1; @@ -1737,24 +1900,69 @@ export int run_tests(std::span passthrough, // for a test that ran in 30ms is not a slow test, it is a // mislabelled one. The phase's own wall time is measured // separately by `tRunPhase` below. + // Nothing to spawn when the invocation already knows the + // answer: the runner is missing, or an earlier spawn was + // refused by the kernel. Recorded as NotRun with that reason. + if (!invocationNotRunReason.empty() || hostCannotRun.load()) { + std::scoped_lock lock(reportMutex); + if (!json) mcpp::ui::plain(std::format("{} ... not run", r.name)); + results.push_back({r.name, TestResult::St::NotRun, 0, {}, {}, 0, false, + invocationNotRunReason.empty() ? hostCannotRunReason + : invocationNotRunReason}); + std::fflush(stdout); + emit_json(results.back()); + continue; + } + const auto tStart = std::chrono::steady_clock::now(); bool timedOut = false; int exitCode = 0; + int spawnErr = 0; std::string runOutput; if (capture) { auto rr = mcpp::platform::process::capture_exec_deadline( - r.argv, r.env, deadline, &timedOut); + r.argv, r.env, deadline, &timedOut, {}, &spawnErr); exitCode = rr.exit_code; runOutput = std::move(rr.output); } else { mcpp::ui::status("Running", std::format("bin/{}", r.name)); exitCode = mcpp::platform::process::run_exec_deadline( - r.argv, r.env, deadline, &timedOut); + r.argv, r.env, deadline, &timedOut, &spawnErr); } auto ms = std::chrono::duration_cast( std::chrono::steady_clock::now() - tStart).count(); std::scoped_lock lock(reportMutex); + if (spawnErr != 0) { + // Refused before it ran (#544). With a runner the failure + // is the runner's own; without one, ENOEXEC is the kernel + // saying this host cannot load the artifact. Either way + // it is a fact about the invocation, so it is printed once + // and every later test is NotRun without a spawn. + using namespace mcpp::build::runner_lookup; + std::string reason; + if (!runnerTmpl.empty()) + reason = spawn_failed_message(r.argv.front(), spawnErr); + else if (classify(spawnErr) == SpawnClass::Unloadable) + reason = std::format( + "this host cannot execute {} artifacts: {} (error {}); " + "declare [target.{}].runner, or pass --no-runner on a " + "host that can", + ctx->tc.targetTriple, errno_text(spawnErr), spawnErr, + ctx->tc.targetTriple); + else + reason = spawn_failed_message(r.argv.front(), spawnErr); + if (!hostCannotRun.exchange(true)) { + hostCannotRunReason = reason; + if (!json) mcpp::ui::warning(reason); + } + if (!json) mcpp::ui::plain(std::format("{} ... not run", r.name)); + results.push_back({r.name, TestResult::St::NotRun, 0, {}, {}, ms, false, + reason}); + std::fflush(stdout); + emit_json(results.back()); + continue; + } if (timedOut) { if (!json) mcpp::ui::plain(std::format( "{} ... FAIL (timeout after {}s)", r.name, testOpts.timeoutSecs)); @@ -1846,30 +2054,11 @@ export int run_tests(std::span passthrough, auto exe = ctx->outputDir / lu.output; - // A freestanding test image cannot run here either, and the answer is - // the SAME runner `mcpp run` uses — one read point, two callers. - // - // Nothing else about the test model changes, and that is a measured - // result rather than a simplification: semihosting propagates the - // firmware's `main` return value to the emulator's exit code - // (`return 7` → qemu exits 7, verified), so "exit code is the verdict" - // holds on bare metal exactly as it does on the host. An earlier plan - // called for a structured stdout protocol because it assumed there was - // no exit code to read; there is. + // Through the runner resolved once above, or bare. The runner's + // program was located already; only the artifact changes per test. std::vector argv; - { - const auto choice = choose_runner(*ctx); - if (choice.freestanding) { - if (choice.tmpl.empty()) { - std::println(stderr, "error: {}", - mcpp::freestanding::no_runner_message(ctx->tc.targetTriple)); - return 2; - } - argv = mcpp::freestanding::expand(choice.tmpl, exe); - } else { - argv.push_back(exe.string()); - } - } + if (runnerTmpl.empty()) argv.push_back(exe.string()); + else argv = mcpp::freestanding::expand(runnerTmpl, exe); for (auto& a : passthrough) argv.push_back(a); std::vector> childEnv; @@ -1907,14 +2096,22 @@ export int run_tests(std::span passthrough, // 7. Summary. int passed = 0; int failed = 0; + int notRun = 0; + std::string notRunReason; std::vector failures; for (auto& r : results) { if (r.status == TestResult::St::Pass) ++passed; + else if (r.status == TestResult::St::NotRun) { + ++notRun; + if (notRunReason.empty()) notRunReason = r.reason; + } else { ++failed; failures.push_back(r.name); } } summary.passed = passed; summary.failed = failed; + summary.notRun = notRun; + summary.notRunReason = notRunReason; // "build X + run Y" rather than one merged number: on a member whose tests // are cheap but whose link is not, those two are three orders of magnitude @@ -1924,30 +2121,50 @@ export int run_tests(std::span passthrough, static_cast(summary.buildMs) / 1000.0, static_cast(summary.runMs) / 1000.0); + // 1 keeps meaning "a test ran and failed". 2 is "could not establish the + // answer": the code the freestanding no-runner path already returns for + // the same situation, and never 0 — a green exit with N tests not run is + // the reading this repository has recorded as its most frequent false + // pass (#544, D2). + const int rc = failed ? 1 : (notRun ? 2 : 0); + if (json) { std::println("{{\"summary\":{{\"member\":\"{}\",\"passed\":{},\"failed\":{}," + "\"not_run\":{},\"not_run_reason\":\"{}\"," "\"elapsed_ms\":{},\"build_ms\":{},\"run_ms\":{}}}}}", test_json_escape(memberName), passed, failed, + notRun, test_json_escape(notRunReason), summary.elapsedMs, summary.buildMs, summary.runMs); std::fflush(stdout); - return failed == 0 ? 0 : 1; + return rc; + } + + // The count is in the summary line at the same weight as failures, with + // its reason: a quiet skip is read as a pass. First line of the reason + // only — the full text was printed when it was established. + auto counts = std::format("{} passed; {} failed", passed, failed); + if (notRun) { + auto firstLine = notRunReason.substr(0, notRunReason.find('\n')); + counts += std::format("; {} not run ({})", notRun, firstLine); } std::println(""); - if (failed == 0) { + if (rc == 0) { mcpp::ui::status("test result", - std::format("ok. {} passed; 0 failed; finished in {}", passed, timing)); + std::format("ok. {}; finished in {}", counts, timing)); return 0; } mcpp::ui::error(std::format( - "test result: FAILED. {} passed; {} failed; finished in {}", - passed, failed, timing)); - std::println(""); - std::println("failures:"); - for (auto& n : failures) std::println(" {}", n); - // (Each compile failure's diagnostics already printed inline under its - // FAIL line in Phase B — the summary stays a compact name list.) - return 1; + "test result: {}. {}; finished in {}", + failed ? "FAILED" : "NOT RUN", counts, timing)); + if (failed) { + std::println(""); + std::println("failures:"); + for (auto& n : failures) std::println(" {}", n); + // (Each compile failure's diagnostics already printed inline under its + // FAIL line in Phase B — the summary stays a compact name list.) + } + return rc; } // `mcpp clean` driver. diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index a7dc68d7..4382782a 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -663,6 +663,15 @@ export struct BuildContext { // derivation would drift from the first exactly when a resolution rule // changes. Written into `.build_cache`; see BuildCacheEntry::depSourceRoots. std::vector depSourceRoots; + // `/bin` of every installed `[xlings] deps` payload of the + // runtime-owner manifest, in declaration order (#544). Read by + // choose_runner's lookup (mcpp.build.runner_lookup) so a runner may name + // a program the project declared without writing the payload's + // home-and-version path into the manifest. Computed by the same + // resolution `fillXpkgDirs` uses for build programs; a payload that is + // declared but not installed contributes nothing, and the lookup then + // continues to PATH. + std::vector xlingsDepBinDirs; std::filesystem::path outputDir; std::filesystem::path stdBmi; std::filesystem::path stdObject; @@ -8698,6 +8707,20 @@ prepare_build(bool print_fingerprint, } ctx.depSourceRoots = std::move(roots); } + // Where a runner may find the programs this project declared (#544). The + // same resolution `fillXpkgDirs` hands to build programs, kept as + // directories rather than env vars because the reader is mcpp's own + // lookup, not a child process. See BuildContext::xlingsDepBinDirs. + if (!runtimeOwnerManifest.xlings.deps.empty()) { + if (auto cfg = get_cfg()) { + auto xlEnv = mcpp::config::make_xlings_env(**cfg); + for (auto const& spec : runtimeOwnerManifest.xlings.deps) { + auto ref = mcpp::xlings::paths::parse_xpkg_ref(spec); + if (auto dir = mcpp::xlings::paths::xpkg_payload(xlEnv, ref)) + ctx.xlingsDepBinDirs.push_back(*dir / "bin"); + } + } + } // ─── Prebuilt dependencies: check before planning to link them ───── // // Here rather than at each place a dependency manifest is loaded, because diff --git a/src/build/runner_lookup.cppm b/src/build/runner_lookup.cppm index 5b86b293..5b778c7b 100644 --- a/src/build/runner_lookup.cppm +++ b/src/build/runner_lookup.cppm @@ -101,10 +101,12 @@ inline std::string errno_text(int e) { inline std::string not_found_message(std::string_view triple, std::string_view argv0, std::span searched) { + // The first line stands on its own: `mcpp test` repeats it in its summary. std::string dirs; for (auto const& d : searched) dirs += "\n " + d.string(); return std::format( - "runner '{}' for '{}' was not found. Searched:{}\n" + "runner '{}' for '{}' was not found on any search path.\n" + " Searched:{}\n" " Declare the package that provides it under [xlings] deps, or " "install it on PATH.\n" " Pass --no-runner to execute the artifact directly on this host.", diff --git a/src/cli.cppm b/src/cli.cppm index 6dc3e597..06380217 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -59,7 +59,7 @@ void print_usage() { std::println(" mcpp new Create a new package skeleton"); std::println(" mcpp build [options] Build the current package"); std::println(" mcpp run [target] [-- args...] Build + run a binary target"); - std::println(" mcpp test [pattern] [-- args...] Build + run tests/**/*.cpp (--list, --timeout, --build-timeout, --message-format json)"); + std::println(" mcpp test [pattern] [-- args...] Build + run tests/**/*.cpp (--list, --timeout, --build-timeout, --message-format json, --no-runner)"); std::println(" mcpp clean [--bmi-cache] Remove target/ (and optionally the build cache)"); std::println(" mcpp add [ns.]pkg@ver Add an exact dependency to mcpp.toml"); std::println(" mcpp remove [ns.]pkg Remove an exact dependency from mcpp.toml"); @@ -391,6 +391,8 @@ int run(int argc, char** argv) { .help("Global dependency cache: global (default) | local | off")) .option(cl::Option("no-cache") .help("Deprecated alias for --cache=off (also clears the build dir)")) + .option(cl::Option("no-runner") + .help("Execute the artifact directly, ignoring any [target.].runner (a host that runs it natively)")) .action(wrap_rc([&passthrough](const cl::ParsedArgs& p) { return cmd_run(p, std::span(passthrough)); }))) @@ -404,6 +406,8 @@ int run(int argc, char** argv) { .help("Output format: human (default) | json (NDJSON, one record per test)")) .option(cl::Option("list") .help("List (filtered) tests without building or running them")) + .option(cl::Option("no-runner") + .help("Run test binaries directly, ignoring any [target.].runner (a host that runs them natively)")) .option(cl::Option("timeout").takes_value().value_name("SECS") .help("Kill a test still RUNNING after SECS seconds (default 300; 0 = no limit)")) .option(cl::Option("build-timeout").takes_value().value_name("SECS") diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index 650f9df8..196383ee 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -213,8 +213,12 @@ export int cmd_run(const mcpplibs::cmdline::ParsedArgs& parsed, std::string target_triple; if (auto tt = parsed.value("target")) target_triple = *tt; if (auto tt = parsed.value("target-triple")) target_triple = *tt; + // --no-runner: "this host can execute the artifact" is a fact about the + // host, and the manifest has no host axis to state it on (#544, D3). + const bool no_runner = parsed.is_flag_set("no-runner"); return mcpp::build::build_run_target(targetName, passthrough, package_filter, - cache_mode, no_cache, target_triple); + cache_mode, no_cache, target_triple, + no_runner); } export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, @@ -238,6 +242,7 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, mcpp::build::TestOptions to; if (parsed.positional_count() > 0) to.filter = parsed.positional(0); to.list = parsed.is_flag_set("list"); + to.noRunner = parsed.is_flag_set("no-runner"); // see cmd_run // The three deadlines share one parser: they differ only in what they // bound, not in how they are spelled. 0 always means "no limit" — for // --timeout that now has to be asked for rather than being the default. @@ -280,9 +285,10 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, int rc = 0; std::vector failed; - std::vector notRun; + std::vector notRun; // --workspace-timeout reached + std::vector unrunnable; // tests built, none executed (#544) std::vector> memberTimes; - int totalPassed = 0, totalFailed = 0; + int totalPassed = 0, totalFailed = 0, totalNotRun = 0; auto tWs = std::chrono::steady_clock::now(); auto ws_ms = [&tWs] { return std::chrono::duration_cast( @@ -309,9 +315,19 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, int r = mcpp::build::run_tests(passthrough, mo, to, &sum); totalPassed += sum.passed; totalFailed += sum.failed; + totalNotRun += sum.notRun; memberTimes.emplace_back(mp, sum.elapsedMs); auto secs = static_cast(sum.elapsedMs) / 1000.0; - if (r != 0) { + if (r == 2 && sum.failed == 0 && sum.notRun > 0) { + // Built and not executed (#544): the member did not fail, and + // it did not pass. 2 outranks 0 and yields to 1, as it does + // for a single member. + if (rc == 0) rc = 2; + unrunnable.push_back(mp); + mcpp::ui::status("Workspace", + std::format("member '{}' ({}/{}) NOT RUN — {} passed, {} not run in {:.2f}s", + mp, idx, members->size(), sum.passed, sum.notRun, secs)); + } else if (r != 0) { rc = r; failed.push_back(mp); mcpp::ui::status("Workspace", @@ -344,10 +360,16 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, } return s; }; + // `not_run` keeps its meaning (members the --workspace-timeout + // stopped before they started); `tests_not_run` and + // `unrunnable_members` are #544's — tests that were built and not + // executed, and the members all of whose tests were. std::println("{{\"workspace_summary\":{{\"members\":{},\"passed\":{},\"failed\":{}," - "\"failed_members\":[{}],\"not_run\":[{}],\"elapsed_ms\":{}}}}}", - members->size(), totalPassed, totalFailed, - join(failed), join(notRun), wsElapsed); + "\"tests_not_run\":{}," + "\"failed_members\":[{}],\"unrunnable_members\":[{}]," + "\"not_run\":[{}],\"elapsed_ms\":{}}}}}", + members->size(), totalPassed, totalFailed, totalNotRun, + join(failed), join(unrunnable), join(notRun), wsElapsed); std::fflush(stdout); return rc; } @@ -370,19 +392,27 @@ export int cmd_test(const mcpplibs::cmdline::ParsedArgs& parsed, for (auto& f : v) { if (!s.empty()) s += ", "; s += f; } return s; }; - if (failed.empty() && notRun.empty()) + // The not-run count is in the line whenever it is non-zero, at the + // same weight as the failure count (#544): a member whose tests were + // built and not executed must not read as a passing member. + std::string notRunCounts = totalNotRun + ? std::format("; {} not run", totalNotRun) : std::string{}; + if (failed.empty() && notRun.empty() && unrunnable.empty()) mcpp::ui::status("workspace result", - std::format("ok. {} member(s); {} passed; 0 failed; finished in {:.2f}s", - members->size(), totalPassed, + std::format("ok. {} member(s); {} passed; 0 failed{}; finished in {:.2f}s", + members->size(), totalPassed, notRunCounts, static_cast(wsElapsed) / 1000.0)); else mcpp::ui::error(std::format( - "workspace test: {}/{} member(s) failed; {} passed; {} failed; " + "workspace test: {}/{} member(s) failed; {} passed; {} failed{}; " "finished in {:.2f}s", - failed.size(), members->size(), totalPassed, totalFailed, + failed.size(), members->size(), totalPassed, totalFailed, notRunCounts, static_cast(wsElapsed) / 1000.0)); if (!failed.empty()) mcpp::ui::plain(std::format(" failed members: {}", join_names(failed))); + if (!unrunnable.empty()) + mcpp::ui::plain(std::format(" not run (no runner on this host): {}", + join_names(unrunnable))); if (!notRun.empty()) mcpp::ui::plain(std::format( " not run (--workspace-timeout {}s reached): {}", diff --git a/tests/e2e/330_runner_hosted_targets.sh b/tests/e2e/330_runner_hosted_targets.sh new file mode 100644 index 00000000..bfc4ef6e --- /dev/null +++ b/tests/e2e/330_runner_hosted_targets.sh @@ -0,0 +1,160 @@ +#!/usr/bin/env bash +# requires: unix-shell +# 330_runner_hosted_targets.sh — `[target.].runner` on a hosted target, +# and what `mcpp run` / `mcpp test` say when this host cannot execute an +# artifact (#544). +# +# Design: .agents/docs/2026-09-02-runner-beyond-baremetal-design.md, §11 lists +# the criteria this file implements. Two properties of the setup carry the +# whole test: +# +# 1. No emulator is needed and none is used. The runner is a shell script +# that records its argv and then executes it, so "the artifact went +# through the runner" is asserted on the exact argv, not on a message. +# 2. The "artifact this host cannot execute" is a real host binary whose ELF +# e_machine is patched to 0xffff AFTER the build. No binfmt_misc entry +# and no native loader accepts it, so posix_spawnp answers ENOEXEC on +# every Linux host — one with qemu-user registered included — and the +# criterion does not depend on which machine runs it. +# +# `requires: unix-shell` and nothing else, on purpose: a `requires: gcc` or +# `requires: llvm` guard skips on both CI shards, and a skip exits 0. +# The ELF half is Linux-only (macOS artifacts are Mach-O); the runner half +# runs on every POSIX host. +set -uo pipefail + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +fail() { echo "FAIL: $*"; exit 1; } +MCPP="${MCPP:?set MCPP to the mcpp binary under test}" + +"$MCPP" new app >/dev/null 2>&1 || fail "mcpp new" +cd app +mkdir -p tests +rm -f tests/*.cpp # the scaffold's own smoke test; the counts below are exact +cat > src/main.cpp <<'EOF' +#include +int main() { std::puts("ARTIFACT-RAN"); return 0; } +EOF +cat > tests/one.cpp <<'EOF' +int main() { return 0; } +EOF + +# The host triple as the manifest spells it. Read from the engine rather than +# guessed: `[target.]` is matched against the resolved target. +out=$("$MCPP" build 2>&1) || fail "initial build: $out" +HOST=$(sed -n 's/.*Target \([^ ]*\) → .*/\1/p' <<<"$out" | head -1) +[[ -n "$HOST" ]] || HOST=$(ls target | head -1) +[[ -n "$HOST" ]] || fail "could not determine the host triple from: $out" + +cat > "$TMP/runner.sh" <<'EOF' +#!/bin/sh +printf '%s\n' "$@" >> "$RUNNER_LOG" +exec "$@" +EOF +chmod +x "$TMP/runner.sh" +export RUNNER_LOG="$TMP/runner.log" + +# ── 1. a declared runner is used on a hosted target, on both `run` doors ──── +printf '\n[target.%s]\nrunner = ["%s"]\n' "$HOST" "$TMP/runner.sh" >> mcpp.toml +out=$("$MCPP" run 2>&1) || fail "run through runner: $out" +grep -q "ARTIFACT-RAN" <<<"$out" || fail "artifact output missing: $out" +[[ -s "$RUNNER_LOG" ]] || fail "runner was not invoked (declared under [target.$HOST]): $out" +exe=$(head -1 "$RUNNER_LOG") +[[ "$exe" == */bin/app ]] || fail "runner argv[0] is not the artifact: $exe" +: > "$RUNNER_LOG" +out=$("$MCPP" run 2>&1) || fail "second run (fast path): $out" +[[ -s "$RUNNER_LOG" ]] || fail "the run fast path bypassed the declared runner: $out" + +# ── 2. --no-runner executes directly and says so ─────────────────────────── +: > "$RUNNER_LOG" +out=$("$MCPP" run --no-runner 2>&1) || fail "--no-runner: $out" +grep -q "ARTIFACT-RAN" <<<"$out" || fail "--no-runner lost the program output: $out" +grep -q "no-runner" <<<"$out" || fail "--no-runner printed no note: $out" +[[ ! -s "$RUNNER_LOG" ]] || fail "--no-runner still invoked the runner" + +# ── 3. runner not found: names the program and the search list, exit 2 ───── +sed -i.bak "s|^runner = .*|runner = [\"mcpp-e2e-no-such-runner\"]|" mcpp.toml +out=$("$MCPP" run 2>&1); rc=$? +[[ $rc -eq 2 ]] || fail "missing runner: exit $rc, want 2: $out" +grep -q "runner 'mcpp-e2e-no-such-runner' for '$HOST' was not found" <<<"$out" \ + || fail "missing runner not named: $out" +grep -q "Searched:" <<<"$out" || fail "search list missing: $out" +grep -q "ARTIFACT-RAN" <<<"$out" && fail "artifact ran without its runner: $out" +# `mcpp test`, one test: every test is not-run with that reason, exit 2. +out=$("$MCPP" test 2>&1); rc=$? +[[ $rc -eq 2 ]] || fail "test with missing runner: exit $rc, want 2: $out" +grep -qE "NOT RUN\. 0 passed; 0 failed; 1 not run \(runner 'mcpp-e2e-no-such-runner'" <<<"$out" \ + || fail "test summary with missing runner: $out" +grep -q "one ... not run" <<<"$out" || fail "per-test not-run line missing: $out" +# --no-runner on `test` runs it directly. +out=$("$MCPP" test --no-runner 2>&1) || fail "test --no-runner: $out" +grep -qE "ok\. 1 passed; 0 failed; finished" <<<"$out" || fail "test --no-runner summary: $out" + +# ── 4. no runner declared, artifact this host cannot load (ELF only) ─────── +if [[ "$(uname -s)" == Linux ]]; then + sed -i.bak "/^\[target\.$HOST\]/,/^runner = /d" mcpp.toml + grep -q "runner" mcpp.toml && fail "runner key still present after removal" + "$MCPP" build >/dev/null 2>&1 || fail "rebuild without runner" + bin=$(ls target/*/*/bin/app | head -1) + [[ -f "$bin" ]] || fail "no artifact at target/*/*/bin/app" + printf '\xff\xff' | dd of="$bin" bs=1 seek=18 conv=notrunc status=none + out=$("$MCPP" run 2>&1); rc=$? + [[ $rc -eq 2 ]] || fail "unrunnable artifact: exit $rc, want 2: $out" + grep -q "this host cannot execute" <<<"$out" || fail "unrunnable message missing: $out" + grep -q "Exec format error" <<<"$out" || fail "the kernel's answer is missing: $out" + grep -q "\[target.$HOST\]" <<<"$out" || fail "paste-able key missing: $out" + grep -q 'runner = \["qemu-aarch64-static"\]' <<<"$out" || fail "runner example missing: $out" + grep -q -- "--no-runner" <<<"$out" || fail "escape hatch not mentioned: $out" + # The criterion measured the patched artifact, not a rebuilt one. + [[ "$(od -An -tx1 -j18 -N2 "$bin" | tr -d ' ')" == "ffff" ]] \ + || fail "the artifact was rebuilt under the run; the criterion measured nothing" + + # `mcpp test`: one test (streaming path) and two tests (capturing path). + # Today they report a spawn failure differently; the assertion is the same. + for n in 1 2; do + if [[ $n -eq 2 ]]; then + cat > tests/two.cpp <<'EOF' +int main() { return 0; } +EOF + fi + # A patched binary is newer than its source, so ninja keeps it; the + # sources are touched so this iteration measures freshly built binaries. + touch tests/*.cpp + out=$("$MCPP" test 2>&1) || fail "test build+run before patching ($n): $out" + grep -qE "ok\. $n passed; 0 failed" <<<"$out" || fail "sanity: tests should pass unpatched ($n): $out" + tbins=$(find target -type f -perm -u+x \( -name one -o -name two \)) + [[ $(wc -l <<<"$tbins") -eq $n ]] || fail "expected $n test binaries, found: $tbins" + for t in $tbins; do + printf '\xff\xff' | dd of="$t" bs=1 seek=18 conv=notrunc status=none + done + out=$("$MCPP" test 2>&1); rc=$? + [[ $rc -eq 2 ]] || fail "test unrunnable ($n): exit $rc, want 2: $out" + grep -qE "NOT RUN\. 0 passed; 0 failed; $n not run \(this host cannot execute $HOST artifacts: Exec format error" <<<"$out" \ + || fail "test summary ($n): $out" + [[ $(grep -c " \.\.\. not run" <<<"$out") -eq $n ]] || fail "per-test not-run lines ($n): $out" + # The reason is printed once when established, and once in the summary. + [[ $(grep -c "cannot execute $HOST artifacts" <<<"$out") -eq 2 ]] \ + || fail "reason printed other than once + summary ($n): $out" + for t in $tbins; do + [[ "$(od -An -tx1 -j18 -N2 "$t" | tr -d ' ')" == "ffff" ]] \ + || fail "test binary was rebuilt under the run ($n)" + done + jout=$("$MCPP" test --message-format json 2>/dev/null); jrc=$? + [[ $jrc -eq 2 ]] || fail "json exit ($n): $jrc, want 2" + [[ $(grep -c '"status":"not_run"' <<<"$jout") -eq $n ]] || fail "json not_run records ($n): $jout" + grep -q "\"not_run\":$n," <<<"$jout" || fail "json summary not_run ($n): $jout" + grep -q '"not_run_reason":"this host cannot execute' <<<"$jout" || fail "json summary reason ($n): $jout" + done +fi + +# ── 5. an array-valued typo is reported, and `runner` is in the list ─────── +printf '\n[target.%s]\nrunnerX = ["x"]\n' "$HOST" >> mcpp.toml +out=$("$MCPP" build 2>&1) +grep -q "unsupported key 'runnerX'" <<<"$out" || fail "array typo not reported: $out" +grep -q "Supported keys: cxx_runtime, linkage, runner, sysroot, toolchain" <<<"$out" \ + || fail "runner missing from the supported-keys list: $out" + +echo "PASS: 330_runner_hosted_targets" From e28a6355d7b7bd01571ee0daaac47b1f90ab687d Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 2 Sep 2026 12:06:42 +0800 Subject: [PATCH 05/12] docs: runner on hosted targets, not-run reporting, [xlings] values per host platform (#544) --- docs/05-mcpp-toml.md | 92 ++++++++++++++++++++++++++- docs/11-machine-output.md | 51 +++++++++++++++ docs/13-baremetal.md | 8 +++ docs/15-openkal-cross.md | 19 ++++++ docs/17-the-project-environment.md | 13 ++++ docs/zh/05-mcpp-toml.md | 77 +++++++++++++++++++++- docs/zh/11-machine-output.md | 44 +++++++++++++ docs/zh/13-baremetal.md | 6 ++ docs/zh/15-openkal-cross.md | 16 +++++ docs/zh/17-the-project-environment.md | 9 +++ 10 files changed, 333 insertions(+), 2 deletions(-) diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index c39d0126..39fbc217 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -1086,7 +1086,7 @@ The selector `` has three forms: |---|---|---| | **bare OS alias** | a single OS / family — the concise, common form | `[target.windows]`, `[target.unix]` | | **`cfg(...)` predicate** | a compound condition (arch / env / combinators) | `[target.'cfg(all(linux, not(arch = "aarch64")))']` | -| **exact triple** | one specific target (also carries `toolchain` / `linkage`) | `[target.x86_64-linux-musl]` | +| **exact triple** | one specific target (also carries `toolchain` / `linkage` / `sysroot` / `runner`; see §2.7.3) | `[target.x86_64-linux-musl]` | A selector may carry platform-conditional **dependencies** and **build flags**: @@ -1284,6 +1284,75 @@ different argv (`-bios default` for an OpenSBI boot, `-bios none -semihosting` for a picolibc image) — and an engine that guesses one is an engine the other board has to fight. A board-support package normally supplies it. +### 2.7.3 `runner` on a hosted target (2026.9.2.1+) + +`[target.].runner` applies to every exact triple, not only to bare +metal. A hosted cross artifact — `aarch64-linux-musl` built on an x86_64 +machine — is executable by some hosts (binfmt_misc with qemu-user registered) +and refused by others with `Exec format error`, and which of the two applies is +a property of the machine, not of the triple. mcpp does not predict it. It +either executes the artifact through the runner the project declared, or it +attempts direct execution and reports what the kernel answered. + +```toml +[target.aarch64-linux-musl] +runner = ["qemu-aarch64-static"] +``` + +The rules, for `mcpp run` and `mcpp test` alike: + +- **A declared runner is used.** Its first element is located by mcpp: first in + the `bin/` directory of each payload declared under `[xlings] deps` (§2.13), + then on `PATH`. A bare name on `PATH` resolves to an xvm shim, which answers + for the current SubOS rather than for the package; the payload lookup is what + lets a runner name a program the project declared. +- **A declared runner that cannot be found or started is an error**, with the + program, the directories searched and the errno. There is no fallback to + direct execution: running the artifact under a different interpreter with + different arguments is the failure the key exists to prevent. +- **No runner, and the kernel refuses the artifact:** `mcpp run` reports the + refusal and the key to write, and exits 2. `mcpp test` reports every test as + not run, with the reason once, and exits 2 (§2.7.3.1). +- **`--no-runner`** executes the artifact directly and ignores a declared + runner. It states a fact about this host — the triple is native here — that + the manifest has no axis to carry; a project whose runner was written for + x86_64 developers is still readable on an aarch64 machine. + +Provisioning the emulator through `[xlings] deps` is the form for a CI job or +a project built on one host class. `qemu-user-aarch64` in the index is built +for x86_64 Linux only, and `[xlings] deps` provisions on every host that builds +the project, so the entry is written per platform (§2.13): + +```toml +[xlings] +deps = [{ linux = "qemu-user-aarch64" }] + +[target.aarch64-linux-musl] +runner = ["qemu-aarch64-static"] +``` + +A package the host cannot install is a hard build error, so an entry without +the platform form would make the project unbuildable on macOS and Windows. The +Linux/aarch64 host, where the package does not exist either, passes +`--no-runner`. + +#### 2.7.3.1 `mcpp test` and tests that were not run + +A test whose artifact this host cannot execute has neither passed nor failed. +`mcpp test` reports it as **not run**, prints the reason once when it is +established, repeats the first line of the reason in the summary, and exits 2: + +``` +warning: this host cannot execute aarch64-linux-musl artifacts: Exec format error (error 8); declare [target.aarch64-linux-musl].runner, or pass --no-runner on a host that can +smoke ... not run +error: test result: NOT RUN. 0 passed; 0 failed; 1 not run (this host cannot execute aarch64-linux-musl artifacts: Exec format error (error 8); ...); finished in 0.41s (build 0.39s + run 0.00s) +``` + +Exit code 1 keeps its meaning — a test ran and failed — and 0 means every test +ran and passed. `--message-format json` carries `"status":"not_run"` and a +`reason` on each record, and `not_run` / `not_run_reason` on the summary +record (see [11 — Machine-Readable Output](11-machine-output.md)). + ### 2.8 `[features]` — Features (Cargo-style, additive) ```toml @@ -1820,6 +1889,27 @@ build needs (`make`/`cmake`/`protoc`/…), pin tool versions per project, or set build-time env vars — without hand-editing `.xlings.json`. `[toolchain]` (§2.7) remains the ergonomic shorthand for the compiler; `[xlings.workspace]` is the general form. +**Values per host platform (2026.9.2.1+).** A `deps` entry and a +`[xlings.workspace]` value may be a table keyed by platform, the form xlings' +own `.xlings.json` accepts for `workspace`: + +```toml +[xlings] +deps = ["xim:ninja", { linux = "qemu-user-aarch64" }, { windows = "nasm", default = "yasm" }] + +[xlings.workspace] +gcc = { linux = "15.1.0" } +llvm = { macos = "20", default = "22" } +``` + +The keys are `linux`, `macos`, `windows` and `default`; `macosx` is accepted as +xlings' spelling of `macos`. mcpp resolves the table against the host it runs +on when the manifest is loaded: the host's key wins, `default` is the fallback, +and a table with neither declares nothing on that host — the entry is absent, +not empty. An unknown key is an error rather than a dropped entry. The axis is +the host operating system only; a package that exists for the OS but not for +the architecture is still a provisioning error on that host. + `subos` selects the root project's **local build/run OS environment**. If the key is absent, mcpp uses its initialized, release-verified `McppDefault` SubOS; `subos = "default"` is an explicit `NamedSubos("default")` selection. There is diff --git a/docs/11-machine-output.md b/docs/11-machine-output.md index fb6e3c72..bc9c0b80 100644 --- a/docs/11-machine-output.md +++ b/docs/11-machine-output.md @@ -358,3 +358,54 @@ from the same resolution a build performs, which may fetch packages, install a payload and run a dependency's build program. A client gates on that table *before* running anything, so an omission would be a safety claim that is not true. + +### `mcpp test --message-format json` — the test stream + +``` +mcpp test [pattern] [--workspace] --message-format json +``` + +This stream predates the envelope of §2 and is not wrapped in it: it is NDJSON, +one record per test as each finishes, then one summary record per member. A +`--workspace` run ends with one `workspace_summary` record. The §6 guarantees +apply to it — fields are added and never removed, and a field's meaning never +changes — and the fields below are the contract as of 2026.9.2.1. + +Per test: + +| field | | +|---|---| +| `member` | the workspace member, or `""` outside a workspace | +| `test` | the path-based test name (`tests/00-a/0.cpp` → `00-a/0`) | +| `status` | `pass`, `compile_fail`, `run_fail`, or `not_run` | +| `exit_code` | the test's exit status; `0` for `not_run` | +| `signal` | the signal number when the status encodes one, else `null` | +| `duration_ms` | build+run wall time of this test | +| `timed_out` | `true` when `--timeout` killed it (`run_fail`) | +| `compile_output`, `run_output` | captured diagnostics | +| `reason` | `not_run` only: why, in one sentence; `""` otherwise | + +Summary record, `{"summary": {...}}`: + +| field | | +|---|---| +| `member`, `passed`, `failed` | counts | +| `not_run` | tests that were built and not executed | +| `not_run_reason` | the reason shared by all of them, or `""` | +| `elapsed_ms`, `build_ms`, `run_ms` | wall time, split | + +⚠️ **`not_run` is neither `pass` nor `run_fail`, and the exit code says so +(2026.9.2.1).** A test is `not_run` when this host cannot load its artifact +(`Exec format error` on a cross target with no runner declared), or when the +declared `[target.].runner` could not be found or started. The +condition is a fact about the invocation: it is established once, the +remaining tests are reported `not_run` without being started, and the process +exits **2**. Exit 1 keeps meaning "a test ran and failed"; exit 0 means every +test ran and passed. A client that read the exit code alone as pass/fail must +handle 2, and a client that inferred "everything passed" from `failed == 0` +must also read `not_run`. + +`workspace_summary` adds `tests_not_run` (the sum over members) and +`unrunnable_members` (members all of whose tests were `not_run`), alongside the +existing `not_run` list, which continues to name members the +`--workspace-timeout` stopped before they started. diff --git a/docs/13-baremetal.md b/docs/13-baremetal.md index 8e8a8373..14032086 100644 --- a/docs/13-baremetal.md +++ b/docs/13-baremetal.md @@ -549,6 +549,14 @@ error: no runner is configured for 'riscv64-none-elf' — a freestanding artifac A board-support package normally supplies this so you do not have to. ``` +The key is not specific to bare metal. A hosted cross target — an +`aarch64-linux-musl` artifact on an x86_64 host — takes the same +`[target.].runner`, with a user-mode emulator such as +`qemu-aarch64-static` in place of the system emulator; on such a target an +absent runner is not an error until the kernel refuses the artifact. The rules +for hosted targets, the `--no-runner` escape and the not-run reporting of +`mcpp test` are in [5 — mcpp.toml](05-mcpp-toml.md), §2.7.3. + ## Writing a board-support package A board-support package is an ordinary mcpp package. It declares the emulator diff --git a/docs/15-openkal-cross.md b/docs/15-openkal-cross.md index 6bd4631b..29577e50 100644 --- a/docs/15-openkal-cross.md +++ b/docs/15-openkal-cross.md @@ -190,6 +190,25 @@ runner = ["qemu-system-riscv64", "-machine", "virt", "-nographic", firmware mode to use are board facts, and an engine that guesses one is an engine a different board has to fight. +A hosted cross target takes the same key with a user-mode emulator +(2026.9.2.1). An `aarch64-linux-musl` artifact built on an x86_64 host is +executed through `qemu-aarch64-static` when the project declares it, and the +package that provides the emulator is declared for the hosts that can install +it: + +```toml +[xlings] +deps = [{ linux = "qemu-user-aarch64" }] + +[target.aarch64-linux-musl] +runner = ["qemu-aarch64-static"] +``` + +Without the key, `mcpp run` reports the kernel's refusal (`Exec format error`) +and the key to write, and `mcpp test` reports every test as not run and exits +2. A host that executes the artifact natively passes `--no-runner`. The rules +are in [5 — mcpp.toml](05-mcpp-toml.md), §2.7.3. + ### The Source Is The Same, The Program Is Not "The same source" is a claim about the toolchain and the standard library, and diff --git a/docs/17-the-project-environment.md b/docs/17-the-project-environment.md index c0ac24d7..e0b05330 100644 --- a/docs/17-the-project-environment.md +++ b/docs/17-the-project-environment.md @@ -124,6 +124,19 @@ instead of installing, and names the packages so they can be provisioned out of band — the same two knobs `[toolchain]` honours, for the same reason: an unasked-for download is not something a build decides on a project's behalf. +The declaration is provisioned on every host that builds the project, and a +package the host cannot install is an error, not a skipped entry. A tool that +exists for one host platform only is therefore declared for that platform +(2026.9.2.1): `deps = [{ linux = "qemu-user-aarch64" }]` declares the emulator +on Linux and nothing elsewhere. The keys and the resolution rule are in +chapter 5, §2.13. + +**The runner.** A program under `[xlings] deps` is also where +`[target.].runner` looks first for its first element, before `PATH` +(chapter 5, §2.7.3). The two keys together provision a user-mode emulator on a +CI host and execute a cross-built artifact through it, without the manifest +naming the payload's path. + ## 6. What belongs somewhere else | Need | Where it goes | diff --git a/docs/zh/05-mcpp-toml.md b/docs/zh/05-mcpp-toml.md index e158094e..fa6125e9 100644 --- a/docs/zh/05-mcpp-toml.md +++ b/docs/zh/05-mcpp-toml.md @@ -952,7 +952,7 @@ linkage = "static" |---|---|---| | **裸 OS 别名** | 单个 OS / 族 —— 简洁且常用的形式 | `[target.windows]`、`[target.unix]` | | **`cfg(...)` 谓词** | 复合条件(arch / env / 组合子) | `[target.'cfg(all(linux, not(arch = "aarch64")))']` | -| **精确三元组** | 某个具体目标(同时承载 `toolchain` / `linkage`) | `[target.x86_64-linux-musl]` | +| **精确三元组** | 某个具体目标(同时承载 `toolchain` / `linkage` / `sysroot` / `runner`,见 §2.7.3) | `[target.x86_64-linux-musl]` | 一个选择器可以承载平台条件的**依赖**与**构建 flag**: @@ -1122,6 +1122,63 @@ mcpp **刻意不提供默认 runner**。用哪个模拟器、哪个机器型号 镜像用 `-bios none -semihosting`)—— 引擎一旦猜一个,另一块板就得跟它打架。板级 支持包通常会提供它。 +### 2.7.3 hosted 目标上的 `runner`(2026.9.2.1+) + +`[target.].runner` 对每一个精确三元组生效,不限于裸机。一个 hosted 交叉产物 +—— 在 x86_64 机器上构建的 `aarch64-linux-musl` —— 有的宿主能直接执行(binfmt_misc +注册了 qemu-user),有的宿主以 `Exec format error` 拒绝;属于哪一种是机器的性质,不是 +三元组的性质。mcpp 不预测它:要么通过工程声明的 runner 执行产物,要么尝试直接执行并 +报告内核的回答。 + +```toml +[target.aarch64-linux-musl] +runner = ["qemu-aarch64-static"] +``` + +规则对 `mcpp run` 与 `mcpp test` 相同: + +- **声明了 runner 就使用它。** 其第一个元素由 mcpp 定位:先在 `[xlings] deps`(§2.13) + 声明的每个载荷的 `bin/` 目录里找,再找 `PATH`。`PATH` 上的裸名会命中 xvm shim,而 + shim 按当前 SubOS 而非按包作答;先查载荷,runner 才能直接写工程声明过的程序名。 +- **声明的 runner 找不到或启动不了是错误**,错误里带程序名、搜索过的目录和 errno。 + 不回落到直接执行:让产物在另一个解释器下带着另一组参数运行,正是这个键要防止的 + 失败。 +- **没有 runner 且内核拒绝产物:** `mcpp run` 报告拒绝原因与应当写的键,退出码 2。 + `mcpp test` 把每个测试报告为未运行,原因只打印一次,退出码 2(§2.7.3.1)。 +- **`--no-runner`** 直接执行产物并忽略声明的 runner。它陈述的是关于本机的事实 —— + 这个三元组在本机是原生的 —— 清单没有承载它的轴;为 x86_64 开发者写的 runner 在 + aarch64 机器上仍可用。 + +通过 `[xlings] deps` 装模拟器是 CI 任务或单一宿主类别工程的形态。索引里的 +`qemu-user-aarch64` 只为 x86_64 Linux 构建,而 `[xlings] deps` 在每台构建本工程的 +宿主上都会 provisioning,所以条目按平台写(§2.13): + +```toml +[xlings] +deps = [{ linux = "qemu-user-aarch64" }] + +[target.aarch64-linux-musl] +runner = ["qemu-aarch64-static"] +``` + +宿主装不了的包是硬构建错误,所以不带平台形式的条目会让工程在 macOS 与 Windows 上 +无法构建。同样没有这个包的 Linux/aarch64 宿主传 `--no-runner`。 + +#### 2.7.3.1 `mcpp test` 与未运行的测试 + +产物在本机无法执行的测试既没有通过也没有失败。`mcpp test` 把它报告为**未运行**, +在确立原因时打印一次,在汇总里重复原因的第一行,退出码 2: + +``` +warning: this host cannot execute aarch64-linux-musl artifacts: Exec format error (error 8); declare [target.aarch64-linux-musl].runner, or pass --no-runner on a host that can +smoke ... not run +error: test result: NOT RUN. 0 passed; 0 failed; 1 not run (this host cannot execute aarch64-linux-musl artifacts: Exec format error (error 8); ...); finished in 0.41s (build 0.39s + run 0.00s) +``` + +退出码 1 含义不变 —— 有测试运行并失败;0 表示每个测试都运行并通过。 +`--message-format json` 在每条记录上带 `"status":"not_run"` 与 `reason`,在汇总记录上 +带 `not_run` / `not_run_reason`(见 [11 —— 机器可读输出](11-machine-output.md))。 + ### 2.8 `[features]` —— Feature(Cargo 风格,可加性) #### 表形式 —— 让 feature 贡献的不止是隐含 feature @@ -1555,6 +1612,24 @@ OPENBLAS_NUM_THREADS = "1" host 工具(`make`/`cmake`/`protoc`…)、按项目固定工具版本、或设构建期环境变量——无需手改 `.xlings.json`。`[toolchain]`(§2.7)仍是编译器的便捷简写;`[xlings.workspace]` 是其通用形式。 +**按宿主平台取值(2026.9.2.1+)。** `deps` 的一个条目与 `[xlings.workspace]` 的一个值可以是 +按平台为键的表,即 xlings 自身 `.xlings.json` 对 `workspace` 接受的形式: + +```toml +[xlings] +deps = ["xim:ninja", { linux = "qemu-user-aarch64" }, { windows = "nasm", default = "yasm" }] + +[xlings.workspace] +gcc = { linux = "15.1.0" } +llvm = { macos = "20", default = "22" } +``` + +键为 `linux`、`macos`、`windows` 与 `default`;`macosx` 作为 xlings 对 `macos` 的拼写也被 +接受。mcpp 在加载清单时按运行它的宿主解析这张表:宿主对应的键优先,`default` 兜底,两者 +都没有时该条目在本宿主上不作声明 —— 是缺席,不是空值。未知的键是错误,不是被丢弃的条目。 +这条轴只到宿主操作系统:一个包存在于该 OS 但不存在于该架构时,在那台宿主上仍是 +provisioning 错误。 + `subos` 选择根项目用于 build/run 的**本地开发 OS 环境**。未声明该键时固定使用 mcpp 已初始化、 经 release 验证的 `McppDefault`;`subos = "default"` 则仍是显式的 `NamedSubos("default")`。没有 CLI/环境变量 override,也不会隐式跟随 xlings active/current。 diff --git a/docs/zh/11-machine-output.md b/docs/zh/11-machine-output.md index 2527b93f..db38e6bf 100644 --- a/docs/zh/11-machine-output.md +++ b/docs/zh/11-machine-output.md @@ -316,3 +316,47 @@ mcpp why toolchain [--target ] [--toolchain ] --format json `network`、`write-global-cache` 与 `exec-build-script`:答案来自与构建同一次的 解析,而那可能拉取包、安装载荷、并运行某个依赖的构建程序。客户端是在**运行之前** 读这张表来决定放不放行的,漏报一项就是一句不成立的安全承诺。 + +### `mcpp test --message-format json` —— 测试流 + +``` +mcpp test [pattern] [--workspace] --message-format json +``` + +这条流早于 §2 的信封,也不被信封包裹:它是 NDJSON,每个测试结束时一条记录,随后每个 +成员一条汇总记录。`--workspace` 运行以一条 `workspace_summary` 记录结束。§6 的保证 +对它同样成立 —— 字段只增不减,字段含义不变 —— 下表是 2026.9.2.1 时的契约。 + +每个测试: + +| 字段 | | +|---|---| +| `member` | workspace 成员;workspace 之外为 `""` | +| `test` | 按路径命名的测试名(`tests/00-a/0.cpp` → `00-a/0`) | +| `status` | `pass`、`compile_fail`、`run_fail` 或 `not_run` | +| `exit_code` | 测试的退出状态;`not_run` 时为 `0` | +| `signal` | 状态编码了信号时是信号号,否则 `null` | +| `duration_ms` | 这个测试构建+运行的墙钟时间 | +| `timed_out` | 被 `--timeout` 杀掉时为 `true`(`run_fail`) | +| `compile_output`、`run_output` | 捕获的诊断输出 | +| `reason` | 仅 `not_run`:一句话说明原因;其余为 `""` | + +汇总记录 `{"summary": {...}}`: + +| 字段 | | +|---|---| +| `member`、`passed`、`failed` | 计数 | +| `not_run` | 已构建但没有执行的测试数 | +| `not_run_reason` | 它们共同的原因,或 `""` | +| `elapsed_ms`、`build_ms`、`run_ms` | 墙钟时间,分段 | + +⚠️ **`not_run` 既不是 `pass` 也不是 `run_fail`,退出码也这么说(2026.9.2.1)。** +本机无法加载测试产物(交叉目标未声明 runner 时的 `Exec format error`),或声明的 +`[target.].runner` 找不到、启动不了时,测试为 `not_run`。这是关于整次调用的 +事实:确立一次,其余测试直接报告为 `not_run` 而不再启动,进程以 **2** 退出。退出码 1 +含义不变 —— 有测试运行并失败;0 表示每个测试都运行并通过。只读退出码判 pass/fail 的 +客户端必须处理 2;由 `failed == 0` 推断「全部通过」的客户端还必须读 `not_run`。 + +`workspace_summary` 增加 `tests_not_run`(各成员之和)与 `unrunnable_members`(所有 +测试都 `not_run` 的成员),与既有的 `not_run` 列表并列;后者仍然指 +`--workspace-timeout` 到达时尚未开始的成员。 diff --git a/docs/zh/13-baremetal.md b/docs/zh/13-baremetal.md index 399a439b..15931691 100644 --- a/docs/zh/13-baremetal.md +++ b/docs/zh/13-baremetal.md @@ -492,6 +492,12 @@ error: no runner is configured for 'riscv64-none-elf' — a freestanding artifac A board-support package normally supplies this so you do not have to. ``` +这个键不限于裸机。hosted 交叉目标 —— x86_64 宿主上的 `aarch64-linux-musl` 产物 —— +使用同一个 `[target.].runner`,以 `qemu-aarch64-static` 这类用户态模拟器代替 +系统模拟器;在这类目标上,缺少 runner 在内核拒绝产物之前不是错误。hosted 目标的规则、 +`--no-runner` 出口与 `mcpp test` 的未运行报告见 [5 —— mcpp.toml](05-mcpp-toml.md) +§2.7.3。 + ## 编写板级支持包 板级支持包是一个普通的 mcpp 包。它在 `[xlings] deps` 下声明所需的模拟器,为消费者 diff --git a/docs/zh/15-openkal-cross.md b/docs/zh/15-openkal-cross.md index 60bdfae6..a0c2fa5d 100644 --- a/docs/zh/15-openkal-cross.md +++ b/docs/zh/15-openkal-cross.md @@ -164,6 +164,22 @@ runner = ["qemu-system-riscv64", "-machine", "virt", "-nographic", `sysroot = ""` 选定零 libc 档。使用哪个机器模型与哪种固件模式是板子的事实, 而一个去猜测它的引擎,是另一块板子必须与之搏斗的引擎。 +hosted 交叉目标使用同一个键,配用户态模拟器(2026.9.2.1)。在 x86_64 宿主上构建的 +`aarch64-linux-musl` 产物,在工程声明后通过 `qemu-aarch64-static` 执行;提供模拟器的包 +按能安装它的宿主声明: + +```toml +[xlings] +deps = [{ linux = "qemu-user-aarch64" }] + +[target.aarch64-linux-musl] +runner = ["qemu-aarch64-static"] +``` + +没有这个键时,`mcpp run` 报告内核的拒绝(`Exec format error`)与应当写的键,`mcpp test` +把每个测试报告为未运行并以 2 退出。能原生执行该产物的宿主传 `--no-runner`。规则见 +[5 —— mcpp.toml](05-mcpp-toml.md) §2.7.3。 + ### 源码是同一份,程序不是 「同一份源码」是关于工具链与标准库的断言,而它成立:`import std` 可用, diff --git a/docs/zh/17-the-project-environment.md b/docs/zh/17-the-project-environment.md index 972bc0a5..3964f4ce 100644 --- a/docs/zh/17-the-project-environment.md +++ b/docs/zh/17-the-project-environment.md @@ -100,6 +100,15 @@ create/bootstrap that environment instead of falling back to active/default 安装,并列出包名以便手动供给 —— 与 `[toolchain]` 遵守的是同样两个开关,理由也相同: 一次没被要求的下载,不该由构建替工程决定。 +这份声明在每台构建本工程的宿主上都会供给,宿主装不了的包是错误,不是被跳过的条目。 +只存在于某一个宿主平台的工具因此按平台声明(2026.9.2.1): +`deps = [{ linux = "qemu-user-aarch64" }]` 在 Linux 上声明这个模拟器,在别处什么都不声明。 +键与解析规则见第 5 章 §2.13。 + +**runner。** `[xlings] deps` 下的程序也是 `[target.].runner` 查找其第一个元素 +的首选位置,在 `PATH` 之前(第 5 章 §2.7.3)。两个键合起来,在 CI 宿主上供给用户态模拟器, +并通过它执行交叉构建的产物,而清单不必写出载荷的路径。 + ## 6. 什么该写在别处 | 需求 | 写在哪里 | From 5bfe8cba1d1cd0b341bfebcd502d0ec66ef9d9f9 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 2 Sep 2026 12:08:16 +0800 Subject: [PATCH 06/12] release: 2026.9.2.1 --- mcpp.toml | 2 +- modules/versioning/src/version.cppm | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/mcpp.toml b/mcpp.toml index 49c41888..9a527ee7 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.9.1.1" +version = "2026.9.2.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index 19c99e35..d44d591a 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.9.1.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.9.2.1"; } // namespace mcpp From 4a48111a22a621c2ed509a41694066110ace6ac7 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 2 Sep 2026 12:17:53 +0800 Subject: [PATCH 07/12] =?UTF-8?q?tests:=20the=20ENOEXEC=20unit=20test=20is?= =?UTF-8?q?=20Linux-only=20=E2=80=94=20macOS=20posix=5Fspawnp=20runs=20an?= =?UTF-8?q?=20unloadable=20file=20through=20/bin/sh=20(measured=20on=20CI)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../2026-09-02-runner-beyond-baremetal-design.md | 10 +++++++++- tests/unit/test_process_run_exec.cpp | 16 +++++++++++++--- 2 files changed, 22 insertions(+), 4 deletions(-) diff --git a/.agents/docs/2026-09-02-runner-beyond-baremetal-design.md b/.agents/docs/2026-09-02-runner-beyond-baremetal-design.md index cc2e7dc4..d291ae20 100644 --- a/.agents/docs/2026-09-02-runner-beyond-baremetal-design.md +++ b/.agents/docs/2026-09-02-runner-beyond-baremetal-design.md @@ -609,7 +609,15 @@ If a use appears, the flag is additive. 1. **macOS.** Measure `posix_spawn` against a wrong-architecture Mach-O. `EBADARCH` and `ENOEXEC` must both map to UNRUNNABLE, and the mapping must be - measured, not assumed. + measured, not assumed. Half measured on 2026-09-02 (PR #545, macOS 14 + ARM64 CI): `posix_spawnp` on an executable file whose content is not a + loadable format does not return `ENOEXEC`; it spawns the file through + `/bin/sh`, as `execvp` does, and the shell exits non-zero with + `spawn_error == 0`. The `ENOEXEC` unit test is therefore Linux-only, and + on macOS the "this host cannot execute" typing is reachable only for a + refusal the kernel reports as an error (`EBADARCH` on a real foreign + Mach-O), which remains unmeasured. The implementation maps `EBADARCH` to + UNRUNNABLE where the macro is defined; nothing asserts it yet. 2. **Windows.** Determine whether the launcher must be moved onto `CreateProcess` before the failure can be typed at all. Wine is not evidence for Windows. diff --git a/tests/unit/test_process_run_exec.cpp b/tests/unit/test_process_run_exec.cpp index 524d9c27..633a4dbe 100644 --- a/tests/unit/test_process_run_exec.cpp +++ b/tests/unit/test_process_run_exec.cpp @@ -128,11 +128,20 @@ TEST(RunExec, MissingProgramTypesErrnoWhenCallerAsks) { EXPECT_TRUE(err.empty()) << err; // the caller owns the report } +#if defined(__linux__) +// Linux only, and that is a MEASUREMENT (CI, macOS 14 ARM64, 2026-09-02): on +// macOS posix_spawnp handles ENOEXEC the way execvp does — it runs the file +// through /bin/sh — so the child IS spawned (spawn_error stays 0) and it is the +// shell that exits non-zero. The kernel's own refusal of a wrong-architecture +// Mach-O (EBADARCH) is a different reading and is not what this file +// synthesises. `mcpp run`'s "this host cannot execute" typing therefore fires +// on Linux for any unloadable file and on macOS only for a real foreign +// binary; see the design's open question 1. TEST(RunExec, UnloadableArtifactIsENOEXEC) { // An executable file with no loader magic: the kernel refuses it with - // ENOEXEC. posix_spawnp does not retry through /bin/sh the way execvp - // does, so the errno reaches the caller untouched. This is the reading - // `mcpp run` turns into "this host cannot execute the artifact". + // ENOEXEC. glibc's posix_spawnp does not retry through /bin/sh the way + // execvp does, so the errno reaches the caller untouched. This is the + // reading `mcpp run` turns into "this host cannot execute the artifact". auto dir = std::filesystem::temp_directory_path() / "mcpp-enoexec-test"; std::filesystem::create_directories(dir); auto f = dir / "not-an-elf"; @@ -143,6 +152,7 @@ TEST(RunExec, UnloadableArtifactIsENOEXEC) { EXPECT_EQ(rc, 127); EXPECT_EQ(spawnErr, ENOEXEC); } +#endif TEST(RunExec, SpawnedChildLeavesSpawnErrorZero) { int spawnErr = -1; From b3cf05fbb01647a9b38cb56c2abbb98abf823df4 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 2 Sep 2026 16:34:17 +0800 Subject: [PATCH 08/12] =?UTF-8?q?run/test:=20look=20the=20runner=20up=20by?= =?UTF-8?q?=20the=20canonical=20triple=20=E2=80=94=20the=20driver's=20spel?= =?UTF-8?q?ling=20matched=20on=20Linux=20and=20never=20on=20macOS=20(measu?= =?UTF-8?q?red=20on=20CI)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- ...26-09-02-runner-beyond-baremetal-design.md | 16 ++++++++++---- src/build/execute.cppm | 22 ++++++++++++++++--- tests/e2e/330_runner_hosted_targets.sh | 13 ++++++----- 3 files changed, 39 insertions(+), 12 deletions(-) diff --git a/.agents/docs/2026-09-02-runner-beyond-baremetal-design.md b/.agents/docs/2026-09-02-runner-beyond-baremetal-design.md index d291ae20..8bcdf922 100644 --- a/.agents/docs/2026-09-02-runner-beyond-baremetal-design.md +++ b/.agents/docs/2026-09-02-runner-beyond-baremetal-design.md @@ -298,10 +298,18 @@ For the resolved target, in order: The per-triple key is already the scoping mechanism for "this runner belongs to that cross target": a runner written under `[target.aarch64-linux-musl]` is not -found when the host target is resolved, because the lookup is -`targetOverrides.find(ctx.tc.targetTriple)` and the key is stored canonicalised -(`toml.cppm:1825-1830, :1960`). No new key is needed to express applicability -along the target axis. +found when the host target is resolved, because the key is stored canonicalised +(`toml.cppm:1825-1830, :1960`) and looked up by the resolved target. No new key +is needed to express applicability along the target axis. + +The lookup key is the canonical spelling, `triple::parse(tc.targetTriple)->str()`, +which is the output directory's name and the key every other +`[target.]` reader in `prepare.cppm` resolves. The draft looked up +`tc.targetTriple` as the driver reported it; on a Linux host that is the +canonical spelling and on macOS it is `arm64-apple-darwin24.6.0`, so +`[target.aarch64-macos].runner` matched on Linux hosts and never on macOS +(measured on PR #545's macOS e2e shard, 2026-09-02). The raw spelling stays as +a fallback for a triple the parser does not know. What the key cannot express is the host axis, and this is the limit D3 and D4 respond to: the same triple is foreign on one host and native on another, and diff --git a/src/build/execute.cppm b/src/build/execute.cppm index ebbf0738..af8d2acd 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -472,9 +472,25 @@ RunnerChoice choose_runner(const BuildContext& ctx, bool noRunner = false) { // for `-bios none -semihosting` while debugging, and — on a hosted cross // triple — for naming the user-mode emulator at all. c.tmpl = ctx.manifest.buildConfig.runner; - if (auto it = ctx.manifest.targetOverrides.find(ctx.tc.targetTriple); - it != ctx.manifest.targetOverrides.end() && !it->second.runner.empty()) { - c.tmpl = it->second.runner; + // The manifest key is the CANONICAL spelling — `aarch64-macos`, the name + // of the output directory and the key every other `[target.]` + // reader uses (prepare.cppm resolves overrides by `t.str()`). The + // toolchain's own `targetTriple` is what the driver reported, which on a + // Linux host happens to be the canonical spelling and on macOS is + // `arm64-apple-darwin24.6.0`. Looking up the raw spelling alone matched on + // Linux and never on macOS (measured on CI, 2026-09-02); the raw form is + // kept as a fallback for a triple the parser does not know. + auto lookup = [&](std::string_view key) { + auto it = ctx.manifest.targetOverrides.find(std::string(key)); + return it != ctx.manifest.targetOverrides.end() && !it->second.runner.empty() + ? &it->second : nullptr; + }; + const mcpp::manifest::TargetEntry* entry = nullptr; + if (auto ft = mcpp::toolchain::triple::parse(ctx.tc.targetTriple)) + entry = lookup(ft->str()); + if (!entry) entry = lookup(ctx.tc.targetTriple); + if (entry) { + c.tmpl = entry->runner; c.fromManifest = !ctx.manifest.buildConfig.runner.empty(); } // `--no-runner` is the operator on THIS host stating a host fact the diff --git a/tests/e2e/330_runner_hosted_targets.sh b/tests/e2e/330_runner_hosted_targets.sh index bfc4ef6e..a09b6d39 100644 --- a/tests/e2e/330_runner_hosted_targets.sh +++ b/tests/e2e/330_runner_hosted_targets.sh @@ -42,12 +42,15 @@ cat > tests/one.cpp <<'EOF' int main() { return 0; } EOF -# The host triple as the manifest spells it. Read from the engine rather than -# guessed: `[target.]` is matched against the resolved target. +# The host triple as the manifest spells it: the CANONICAL form, which is the +# name of the output directory (`target//…`) and the key every +# `[target.]` reader resolves. Read from the engine rather than +# guessed. Not the "Target … → …" status line: on macOS that prints the +# driver's own spelling (`arm64-apple-darwin24.6.0`), which is not the key — +# and the first version of this test used it and failed only on macOS. out=$("$MCPP" build 2>&1) || fail "initial build: $out" -HOST=$(sed -n 's/.*Target \([^ ]*\) → .*/\1/p' <<<"$out" | head -1) -[[ -n "$HOST" ]] || HOST=$(ls target | head -1) -[[ -n "$HOST" ]] || fail "could not determine the host triple from: $out" +HOST=$(ls target | head -1) +[[ -n "$HOST" ]] || fail "could not determine the host triple from target/: $out" cat > "$TMP/runner.sh" <<'EOF' #!/bin/sh From 8da71f32b5f25814ae1944fcd777267016893c08 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 2 Sep 2026 21:36:52 +0800 Subject: [PATCH 09/12] run/test: diagnostics name the canonical triple, which is the key the reader wrote MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `choose_runner` resolves `[target.].runner` against the canonical spelling — the output directory's name, and the key every other `[target.]` reader uses. The four diagnostics it feeds printed `tc.targetTriple`, the spelling the driver reported, which on a Linux host is the same string and on macOS is `arm64-apple-darwin24.6.0`. Two consequences, both user-facing. The not-found message named a triple the author never wrote. The unrunnable message printed a `[target.…]` block to paste whose key no lookup would ever match, so following the advice would have left the artifact running bare a second time. `RunnerChoice` now carries `tripleKey`, derived once beside the lookup that uses it, so the two cannot disagree. The triple is parsed once rather than three times. Measured: e2e 330 §3 asserts the message names the triple the manifest key uses and failed only on macOS ARM64 (run 33609434208). --- src/build/execute.cppm | 31 ++++++++++++++++++++----------- 1 file changed, 20 insertions(+), 11 deletions(-) diff --git a/src/build/execute.cppm b/src/build/execute.cppm index af8d2acd..cab3a2ab 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -459,12 +459,22 @@ struct RunnerChoice { bool freestanding = false; // an EMPTY tmpl is fatal when true bool fromManifest = false; // the consumer overrode a dependency's bool ignored = false; // --no-runner dropped a declared template + // The spelling that names this target in the manifest: the canonical form, + // which is also the output directory's name and the key every + // `[target.]` reader resolves. Every diagnostic below prints this + // rather than `tc.targetTriple`, because each one either names the key the + // author wrote or prints a key to paste, and the driver's own spelling is + // neither — on macOS it is `arm64-apple-darwin24.6.0`, which no + // `[target.…]` lookup matches. Derived here, once, so the lookup and the + // message it produces cannot disagree about which target they mean. + std::string tripleKey; }; RunnerChoice choose_runner(const BuildContext& ctx, bool noRunner = false) { RunnerChoice c; - if (auto ft = mcpp::toolchain::triple::parse(ctx.tc.targetTriple)) - c.freestanding = ft->is_freestanding(); + const auto ft = mcpp::toolchain::triple::parse(ctx.tc.targetTriple); + if (ft) c.freestanding = ft->is_freestanding(); + c.tripleKey = ft ? ft->str() : ctx.tc.targetTriple; // Two producers, ordinary precedence: what the author of THIS project // wrote beats what a dependency supplied. The dependency is the normal // case on bare metal (a board-support package computes the emulator's @@ -485,10 +495,9 @@ RunnerChoice choose_runner(const BuildContext& ctx, bool noRunner = false) { return it != ctx.manifest.targetOverrides.end() && !it->second.runner.empty() ? &it->second : nullptr; }; - const mcpp::manifest::TargetEntry* entry = nullptr; - if (auto ft = mcpp::toolchain::triple::parse(ctx.tc.targetTriple)) - entry = lookup(ft->str()); - if (!entry) entry = lookup(ctx.tc.targetTriple); + const mcpp::manifest::TargetEntry* entry = lookup(c.tripleKey); + if (!entry && c.tripleKey != ctx.tc.targetTriple) + entry = lookup(ctx.tc.targetTriple); if (entry) { c.tmpl = entry->runner; c.fromManifest = !ctx.manifest.buildConfig.runner.empty(); @@ -1385,7 +1394,7 @@ export int build_run_target(const std::optional& targetName, if (!found.program) { std::println(stderr, "error: {}", mcpp::build::runner_lookup::not_found_message( - ctx->tc.targetTriple, tmpl.front(), found.searched)); + choice.tripleKey, tmpl.front(), found.searched)); return 2; } tmpl.front() = found.program->string(); @@ -1429,7 +1438,7 @@ export int build_run_target(const std::optional& targetName, std::println(stderr, "error: {}", spawn_failed_message(argv.front(), spawnErr)); else if (classify(spawnErr) == SpawnClass::Unloadable) std::println(stderr, "error: {}", - unrunnable_message(ctx->tc.targetTriple, exe, spawnErr)); + unrunnable_message(choice.tripleKey, exe, spawnErr)); else std::println(stderr, "error: {}", spawn_failed_message(exe.string(), spawnErr)); return 2; @@ -1878,7 +1887,7 @@ export int run_tests(std::span passthrough, runnerTmpl.front(), ctx->xlingsDepBinDirs, pathEnv ? pathEnv : ""); if (found.program) runnerTmpl.front() = found.program->string(); else invocationNotRunReason = mcpp::build::runner_lookup::not_found_message( - ctx->tc.targetTriple, runnerTmpl.front(), found.searched); + runnerChoice.tripleKey, runnerTmpl.front(), found.searched); } // Set by the first worker whose spawn the kernel refused; every worker // checks it before spawning. Workers already past the check may be @@ -1964,8 +1973,8 @@ export int run_tests(std::span passthrough, "this host cannot execute {} artifacts: {} (error {}); " "declare [target.{}].runner, or pass --no-runner on a " "host that can", - ctx->tc.targetTriple, errno_text(spawnErr), spawnErr, - ctx->tc.targetTriple); + runnerChoice.tripleKey, errno_text(spawnErr), spawnErr, + runnerChoice.tripleKey); else reason = spawn_failed_message(r.argv.front(), spawnErr); if (!hostCannotRun.exchange(true)) { From 96cfb7159ab550f4e6d3f4d76b89bf779d869c6b Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 2 Sep 2026 21:39:27 +0800 Subject: [PATCH 10/12] run/test: the two notes and the freestanding error name the key as well MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Six message sites remained on `tc.targetTriple` after the previous commit, and three of them print a `[target.]` key the reader is meant to act on: the override note, its `mcpp test` twin, and `no_runner_message`, which shows a complete key-and-value block to paste. On macOS each named `arm64-apple-darwin24.6.0`, a spelling no `[target.…]` lookup resolves, so following the advice would have produced a key that is never read. The remaining two, the `--no-runner` notes, name the target a runner was declared for, and the declaration is under the canonical key. Every diagnostic on both paths now reads `RunnerChoice::tripleKey`. On Linux the two spellings coincide, which is why e2e 130, 131 and 132 pass unchanged. --- src/build/execute.cppm | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/build/execute.cppm b/src/build/execute.cppm index cab3a2ab..3b31423c 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -1369,14 +1369,14 @@ export int build_run_target(const std::optional& targetName, const auto choice = choose_runner(*ctx, no_runner); if (choice.ignored) mcpp::ui::info("note", std::format( - "--no-runner: ignoring the runner declared for {}", ctx->tc.targetTriple)); + "--no-runner: ignoring the runner declared for {}", choice.tripleKey)); if (choice.fromManifest) mcpp::ui::info("note", std::format( "[target.{}].runner overrides the runner a dependency supplied", - ctx->tc.targetTriple)); + choice.tripleKey)); if (choice.freestanding && choice.tmpl.empty()) { std::println(stderr, "error: {}", - mcpp::freestanding::no_runner_message(ctx->tc.targetTriple)); + mcpp::freestanding::no_runner_message(choice.tripleKey)); return 2; } if (!choice.tmpl.empty()) { @@ -1868,14 +1868,14 @@ export int run_tests(std::span passthrough, const auto runnerChoice = choose_runner(*ctx, testOpts.noRunner); if (runnerChoice.ignored && !json) mcpp::ui::info("note", std::format( - "--no-runner: ignoring the runner declared for {}", ctx->tc.targetTriple)); + "--no-runner: ignoring the runner declared for {}", runnerChoice.tripleKey)); if (runnerChoice.fromManifest && !json) mcpp::ui::info("note", std::format( "[target.{}].runner overrides the runner a dependency supplied", - ctx->tc.targetTriple)); + runnerChoice.tripleKey)); if (runnerChoice.freestanding && runnerChoice.tmpl.empty()) { std::println(stderr, "error: {}", - mcpp::freestanding::no_runner_message(ctx->tc.targetTriple)); + mcpp::freestanding::no_runner_message(runnerChoice.tripleKey)); return 2; } std::vector runnerTmpl = runnerChoice.tmpl; From e9f448dd90134cf91674c181ed2e28a692b4b2fe Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 2 Sep 2026 21:41:24 +0800 Subject: [PATCH 11/12] CHANGELOG: 2026.9.2.1 --- CHANGELOG.md | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index b32343da..2ce4e6ec 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,79 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 +## [2026.9.2.1] — 2026-09-02 + +`[target.].runner` 对每一个目标生效,启动失败不再无声,`mcpp test` 把跑不起来 +的测试报成 not run 并退 2,`[xlings]` 的值可以按宿主平台给出。 + +设计与实测见 +[`.agents/docs/2026-09-02-runner-beyond-baremetal-design.md`](.agents/docs/2026-09-02-runner-beyond-baremetal-design.md)。 + +> **一个键被解析、被类型检查、被文档记录,而读它的那处代码在读之前就返回了。** +> `choose_runner` 在 `os == "none"` 之外一律返回空模板,于是宿主交叉目标 +> (x86_64 上构建的 `aarch64-linux-musl`)的 runner 从未被查询过:`mcpp run` 裸执行 +> 产物,内核以 `ENOEXEC` 拒绝,而 `run_exec` 把这次拒绝变成一个不打印任何东西的 +> 127。同一层的有界启动器在自己的声明里写着「『起不来』与『跑了但失败』不能共用 +> 一个退出码」,而它的两个调用方都靠**再 spawn 一次**来回落,把第一次的 errno 丢掉。 + +### 修复 + +- **`[target.].runner` 对每个目标生效。** freestanding 谓词只决定一件事: + 没有 runner 时是否在任何 spawn 之前就失败。宿主能不能执行一个外来 ISA 的产物不再 + 被预测 —— 这台机器上 `binfmt_misc` 注册了 qemu-user 就能跑,报告 #544 的那台不能, + 而两者的三元组相同。mcpp 要么执行工程声明的 runner,要么尝试启动并报告内核的回答。 + +- **runner 的程序由 mcpp 定位,不由 `posix_spawnp` 定位。** 先在 `[xlings] deps` + 声明的每个载荷的 `bin/` 里找,再走 `PATH`。裸名在 `PATH` 上解析到的是 xvm 垫片, + 而垫片按**当前 SubOS** 回答而不是按包所在的位置回答(e2e 130 在 CI 里记录过这一条: + 同一个 job 里 `qemu-system-riscv64 --version` 成功,而 `mcpp run` 执行同一个裸名 + 得到「未安装」)。「哪儿都找不到」在任何 spawn 之前判定,并且是错误而不是回落到 + 裸执行:让产物在另一个解释器下带着另一组参数运行,正是这个键存在要防止的失败。 + +- **启动失败被定型并且只被报告一次。** `DeadlineRun` 与 `BoundedOutcome` 带上 + spawn 错误码;`run_exec`、`capture_exec` 与两个 deadline 包装各多一个末位 + `int* spawn_error`。调用方要了错误码就由调用方报告,没要就由启动器自己报告。 + 没有第二次 spawn。 + +- **`mcpp test` 有了第四种状态。** 产物这台宿主装载不了的测试既没有通过也没有失败。 + 它被报成 **not run**,原因在确立时打印一次,在汇总行里以与失败同等的分量再出现一次, + 退出码是 2 —— freestanding 的 no-runner 路径对同一种处境早就用这个码。1 保持 + 「跑了并且失败」,0 保持「每个测试都跑了并且通过」。`--message-format json` 带上 + `"status":"not_run"` 与每条记录的 `reason`,汇总记录带上 `not_run` / + `not_run_reason`;`--workspace` 另加 `tests_not_run` 与 `unrunnable_members`。 + +- **`--no-runner`,`mcpp run` 与 `mcpp test` 都接受。** 「这台宿主能直接执行该产物」 + 是关于宿主的事实,而 manifest 没有宿主轴:`[target.]` 按目标索引, + `[xlings] deps` 没有任何索引。为 x86_64 开发者写的 runner 在 aarch64 宿主上同样会 + 被读到,而那里模拟器既无用也装不上。旗标由那台宿主上的操作者给出,因为只有那里 + 知道这件事。 + +- **`[xlings]` 的值可以按宿主平台给出。** `deps` 的一项与 `[xlings.workspace]` 的一个 + 值都可以写成 `{ linux = "...", macos = "...", windows = "...", default = "..." }` —— + xlings 自己的 `.xlings.json` 对 `workspace` 接受的就是这个形式,`macosx` 作为它的 + 拼法一并接受。在 manifest 加载时对本机解析,因此下游每一个读者看到的仍然是一张平表。 + 未知的平台键是硬错误。在此之前 `[xlings.workspace]` 会**静默丢掉**一个表值,而 + `[xlings] deps` 根本没有条件化形式 —— 这让 `deps = ["qemu-user-aarch64"]` + (索引里只为 x86_64 构建的包)在其余每一类宿主上都是硬构建错误。 + +- **`[target.]` 的未知键普查覆盖数组。** `runnerX = ["x"]` 会被报出来, + 而支持键的清单里补上了 `runner` —— 此前普查刻意跳过数组,代价是这张表读取的唯一 + 一个数组键既不在清单里,拼错了也无人报告。 + +- **诊断名出的是规范拼法。** 六处消息此前打印驱动报出的三元组;其中三处打印的是 + 一段供粘贴的 `[target.]`,而在 macOS 上那是 `arm64-apple-darwin24.6.0`, + 没有任何 `[target.…]` 查询会命中它。`RunnerChoice::tripleKey` 与查询本身在同一处 + 求出,两者不可能各说各的。 + +### 行为变化 + +- 宿主三元组下已经声明的 `runner` 从此生效。在其程序缺失的宿主上,`mcpp run` / + `mcpp test` 现在带消息失败,而不是静默裸跑产物;`--no-runner` 是出口。 +- 此前对跑不起来的产物报 `FAIL (exit 127)` 的 `mcpp test` 现在报 `NOT RUN` 并退 2 + (仍然非零,CI 作业不会因此改变颜色)。 +- `[xlings] deps` 与 `[xlings.workspace]` 接受表值;带未知平台键的表此前被丢弃, + 现在是错误。 + ## [2026.9.1.1] — 2026-09-01 #540 的七条审计,加上核验它们时挖出的四条没有人报过的。它们几乎全是同一族: From e5a81dfbe5e7b6490ab24d5a8492e9886df95405 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Wed, 2 Sep 2026 21:42:51 +0800 Subject: [PATCH 12/12] design: state where the payload-bin rule is actually covered --- ...26-09-02-runner-beyond-baremetal-design.md | 24 +++++++++++++++---- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/.agents/docs/2026-09-02-runner-beyond-baremetal-design.md b/.agents/docs/2026-09-02-runner-beyond-baremetal-design.md index 8bcdf922..289749a4 100644 --- a/.agents/docs/2026-09-02-runner-beyond-baremetal-design.md +++ b/.agents/docs/2026-09-02-runner-beyond-baremetal-design.md @@ -708,11 +708,25 @@ the bounded launcher reports a spawn failure. Section 1 shows these are one defect seen from two sides; a test that covers only the untimed path leaves the doubled spawn in place. -**What is not exercised.** Provisioning `qemu-user-aarch64` through -`[xlings] deps` is not an e2e criterion: it needs network, an x86_64 host, -and a registry, and the provisioning pass has its own coverage from #531. The -payload-`bin/` rule is exercised with a locally staged package directory -instead (third row). +**What the e2e does not exercise, and what does.** Provisioning +`qemu-user-aarch64` through `[xlings] deps` is not an e2e criterion: it needs +network, an x86_64 host and a registry, and the provisioning pass has its own +coverage from #531. Neither is the payload-`bin/` rule, and the third row of +the table above is therefore not an e2e row. Staging a package into the store +to obtain one was considered and rejected: the store is shared with the +machine's real payloads, and a declared package that is not installable is a +hard build error, so a synthetic entry cannot be declared without the +provisioning pass refusing it first. + +The rule is covered on two levels instead. `locate`'s ordering — a payload +`bin/` before `PATH`, a non-executable file skipped, every directory recorded +— is asserted directly in `tests/unit/test_runner_lookup.cpp`. The wiring from +`[xlings] deps` through `BuildContext::xlingsDepBinDirs` to that call is +asserted by the sandbox verification (section 13), where `qemu-aarch64-static` +resolves through the declared payload while the same bare name on `PATH` is an +xvm shim that answers for the current SubOS. That verification is the only +place all three parts are present at once, and it is a required step of the +release rather than an optional one. ## 12. Implementation surface