Skip to content

Five open issues, three of them misdiagnosed by their reports, and staging as a service (2026.9.11.2) - #607

Merged
Sunrisepeak merged 6 commits into
mainfrom
fix/staging-is-a-service-for-a-dispatched-format
Sep 11, 2026
Merged

Five open issues, three of them misdiagnosed by their reports, and staging as a service (2026.9.11.2)#607
Sunrisepeak merged 6 commits into
mainfrom
fix/staging-is-a-service-for-a-dispatched-format

Conversation

@Sunrisepeak

@Sunrisepeak Sunrisepeak commented Sep 10, 2026

Copy link
Copy Markdown
Member

Two goal documents, one PR, one version. Commits are the checkpoints.

.agents/docs/2026-09-11-six-open-issues-analysis.md -- the analysis, the
self-review that changed one of its plans, and the criteria each fix has to
meet. .agents/docs/2026-09-11-distribution-plugins-and-platform-decomposition.md
section 11 -- the implemented distribution work read from nine angles.

Every report was right about its symptom; three were wrong about the cause

#606 -- the scanner read inside comments, in both directions. Block-comment
state never existed in the file's history: git log -S over
src/modgraph/scanner.cppm returns no commit that ever added one. So the
bisect to "after 2026.9.7.1" dates when the defect became audible, not when
it was introduced, and the malformed-name refusal added at 2026.9.9.1 (#594) is
correct.

The unreported form is worse. export module y; inside a block comment made a
plain .cpp the recorded producer of gcm.cache/y.gcm, a BMI the compiler
never writes -- and a file that legitimately imports y was then told
imports must be built before being imported, an ordering problem that does not
exist, while the real provider was never searched for.

And the other direction, also unreported: a commented-out raw-string opener
put the raw-string pass into a state it could not leave, blanking every
following line until a )" that never comes. Real imports were invisible to
the scanner and visible to the compiler -- a missing dependency edge, which
is a build-order race rather than a deterministic refusal.

The root cause is an argument written down as settled, in the source: "Ordinary
"..." strings are intentionally left as-is: the import/module matcher only
fires on lines whose trimmed text starts with the keyword, which a string body
can only do when it spans lines (i.e. a raw string)."
The premise holds and the
enumeration is short by one -- a block comment can do it too.

One pass over three states now. tests/e2e/639 holds eight criteria, two of
them properties that were already correct and must not be lost (/* */ import x; must still record the import; "a /* b" must not open a comment). Against
released 2026.9.10.2, six of the eight go red.

#604 -- a two-token switch lost its switch. The host-module flag collector
de-duplicated per token and was written for the one family whose marker is a
single idempotent word:

family useFlags tokens
GCC -fmodules 1, idempotent
Clang -fmodule-file=<name>=<path> 1, unique
MSVC /reference, <name>=<path> 2, first one repeats

/reference was already in the list from the bundled mcpp module, so an inner
host module's pair lost its first half and cl read <name>=<path> as a source
file name (C1083). Clang is immune by construction, which is why the defect was
specific to the one toolchain selection that reaches import std; at c++20 on
Windows.

Appended verbatim now, de-duplicating nothing. The comment being replaced stated
the filter's whole value -- "repeating it is harmless but noisy" -- so it bought
a tidier argv and paid with a broken command line. Plus
mcpp::toolchain::orphaned_reference as a reader that refuses before the
command runs, because C1083 names the module and the BMI and never names the
missing switch.

#603 -- the level is a property of the STL. The report's fix needs one
correction: std_module_min_level(tc) reads tc.version, which on the clang
path is clang's, against an MSVC threshold -- clang 20.x would pass it by
accident and clang 19.x fail it wrongly. The binding version is in the path of
the std.ixx already selected. One function, both paths, and a test that the
two forms agree for a well-formed installation.

#564 -- two dead keys wanting opposite answers. default_jobs gained a
reader: it is the only level that can hold a machine fact, and it also bounds
mcpp test (documented, because one key with two behaviours has to be).
default_backend was removed: BackendKind has two values and src/build/ has
one backend. The test asserts the precedence, not the wiring -- a fixture that
only sets the global value would pass with the parameter wired above
MCPP_JOBS.

#599 -- a check that had run in zero CI jobs. Three defects, not two: the
hub path was written for the current layout while naming a historical tree, the
uninit branch printed a note that cannot turn a job red, and there is no
bench workflow
-- so the test's own justification for the note was false. All
three fixed; the note now fails under CI=true and stays a note locally.
Verified in both directions.

Staging is a service, not a precondition (the original subject)

mcpp pack --format <name> staged unconditionally before dispatching and let a
staging failure fail the command. For tar / dir that is right -- the staged
tree is the artifact. For a dispatched format it is one input the provider may
or may not want, and treating it as a precondition made every dispatched format
unreachable on any target whose built-in packaging is refused. Measured on
macos-15: --format app never reached the dispatch, because the built-in
closure walk uses LD_TRACE_LOADED_OBJECTS and dyld answers it by running the
program
. The engine was answering a question the provider was not asked.

Verification

109 unit tests pass, tests/e2e/638 (nine cases) and tests/e2e/639 (eight)
pass, tests/e2e/233 passes with the submodules checked out and fails when they
are absent under CI. 266_pack_refuses_a_macho_program A/B'd byte-identical
against released 2026.9.10.2.

…or dispatch (2026.9.11.2)

`mcpp pack --format <name>` staged unconditionally before dispatching, and a
staging failure failed the whole command. For `--format tar` and `--format dir`
that is right -- the staged tree IS the product. For a DISPATCHED format it is
an input the provider may or may not want, and treating it as a precondition
made EVERY dispatched format unreachable on any target whose built-in bundling
is refused.

Measured on macos-15 with 2026.9.11.1: `mcpp pack --format app` never reached
the dispatch at all, because `pack::run` refuses a Mach-O PROGRAM outright --
the built-in closure walk is `LD_TRACE_LOADED_OBJECTS`, which is glibc's, and
dyld ignores it and simply runs the program. That refusal is correct about the
built-in archive and says nothing about whether a `.app` bundler can work: a
bundler that names one program needs no closure walk. THE ENGINE WAS ANSWERING
A QUESTION THE PROVIDER HAD NOT BEEN ASKED.

The failure is now reported and CARRIED rather than fatal or swallowed. The
reason is printed as a warning, `pack_stage_dir` stays empty, and
`${mcpp.stage_dir}` refuses at expansion naming that reason. A provider that
reads the tree gets a precise diagnostic; one that does not proceeds. Nothing
is silently degraded -- what changes is who decides.

`BuildOverrides::pack_stage_reason` is that channel. Without it a build that is
plainly packaging would read "this build is not packaging", which sends a member
author looking in the wrong place -- the refusal has to distinguish "you did not
ask for a format" from "no tree could be staged for this target".

`638_pack_format_dispatch.sh` gains the case macOS found, held on every
platform. Linux cannot reproduce the Mach-O refusal, so the leg asserts the
property the fix rests on instead: a provider that reads no staged tree is
dispatched and reported, and gains no dependency on the stage manifest. Nine
cases now, each paired with the wrong answer it excludes.

WHAT FOUND THIS IS WORTH RECORDING. `dist-apple` had only plan-level assertions
behind it until CI ran it on a real macOS runner for the first time. A
plan-level assertion says the gate is right and says nothing about whether the
tool accepts what the member renders -- which is why the two platform-specific
CI steps were added, and why the first thing they did was fail.

109 unit tests pass. `266_pack_refuses_a_macho_program` still fails on this
machine and was A/B'd against released 2026.9.10.2: byte-identical logs, so
this change does not touch it. Plain `mcpp pack` keeps the fatal refusal, since
`opts.format != Dispatched` there.
Section 9 reviewed the proposal on its own terms; section 11 reviews what
shipped, from the nine angles a reviewer applies independently of it. Each
angle is recorded with the wrong answer it excluded and, where one exists, the
measurement that changed the design -- the architecture section names the two
occurrences of the same error (the engine deciding on the provider's behalf),
the test-coverage section names the CI assertion that was itself the defect,
and the last section gives the dependency order and why it is not the order the
rows land in.
…s, and three of the reports were wrong about the cause

#606  THE SCANNER READ INSIDE COMMENTS, IN BOTH DIRECTIONS. Block-comment
      state never existed in the file's history -- `git log -S` returns no such
      commit -- so the bisect to "after 2026.9.7.1" dates when the defect
      became audible, not when it was introduced; the refusal added at
      2026.9.9.1 is correct. The unreported form is worse: `export module y;`
      inside a block comment made a plain `.cpp` the recorded producer of
      `y.gcm`, and a real importer of `y` was then told `imports must be built
      before being imported`. And the OTHER direction, also unreported: a
      commented-out raw-string opener blanked every following line, so real
      imports were invisible to the scanner and visible to the compiler -- a
      missing dependency edge, which is a build-order race rather than a
      deterministic refusal.

      One pass over three states. Eight criteria in tests/e2e/639, two of them
      properties that were already correct and must not be lost; six of the
      eight go red against released 2026.9.10.2.

#604  A TWO-TOKEN SWITCH LOST ITS SWITCH. The host-module flag collector
      de-duplicated per token and was written for GCC's one-token `-fmodules`.
      MSVC's reference is a pair whose first half legitimately repeats, so the
      pair lost `/reference` and cl read `<name>=<path>` as a source file
      (C1083). Appended verbatim now; the comment being replaced stated the
      filter's whole value ("harmless but noisy"). Plus
      `orphaned_reference` as a reader that names the cause before the command
      runs.

#603  THE LEVEL IS A PROPERTY OF THE STL. Calling the existing probe from
      clang's path would have compared a clang version against an MSVC
      threshold. The toolset version is in the path of the `std.ixx` already
      selected; one function, both paths, and a test that the two forms agree
      for a well-formed installation.

#564  `default_jobs` gained a reader and `default_backend` was removed. Two
      dead keys wanting opposite answers: one names a machine fact with no
      other home, the other promises a choice that does not exist. The test
      asserts the precedence, not the wiring.

#599  A CHECK THAT HAD RUN IN ZERO CI JOBS. Three defects, not the two
      reported: the hub path was written for the current layout while naming a
      historical tree, the `uninit` branch printed a note that cannot turn a
      job red, and there is no bench workflow at all -- so the test's own
      justification for the note was false. All three fixed, and the note now
      fails under CI while staying a note locally.
The comment and the analysis both named `docs/03-configuration.md`, which
does not exist -- caught by this repository's own docs check, which asserts
that every document a file names is present.
@Sunrisepeak Sunrisepeak changed the title Staging is a service to the provider, not a precondition for dispatch (2026.9.11.2) Five open issues, three of them misdiagnosed by their reports, and staging as a service (2026.9.11.2) Sep 10, 2026
…rees

The new criterion fired on macOS and was right to. `submodules: recursive`
went onto the Linux e2e shards only, and `233_bench_matrix.sh` runs in every
job that invokes the whole suite -- of which there are three, not one. That is
the same shape as the defect it was added for: a check reasoned about against
one job and applied to all of them.

Enumerated rather than guessed this time. The three unfiltered
`bash tests/e2e/run_all.sh` jobs are ci-linux-e2e, ci-macos-e2e and
ci-windows-e2e, and all three now check the trees out.
ci-windows-msvc-xlings sets `E2E_ONLY: '239_*.sh'`, so 233 does not run
there; ci-linux-e2e's baremetal job and openkal-cross name their tests
directly.

The failure now says what to do rather than asserting that this job already
asks for the submodules -- which would be false for exactly the job that
needs to be told.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants