Skip to content

fix: a kernel-abi package over the system's C library is placed into the shared image that reaches it - #846

Closed
Sunrisepeak wants to merge 1 commit into
mainfrom
fix/kernel-abi-in-shared-images
Closed

Sunrisepeak wants to merge 1 commit into
mainfrom
fix/kernel-abi-in-shared-images

Conversation

@Sunrisepeak

Copy link
Copy Markdown
Member

The defect

A shared library built on mcpplibs.os imports kal_* instead of defining them. Measured with openmui's CPython extension (kind = "shared" in a workspace member):

ImportError: .../_openmui.abi3.so: undefined symbol: kal_process_close

The cause is make_plan's static placement (#646 F1). Every package that provides an mcpp: layer is a boundary: it is never placed into a shared image, and its objects go into the programs only. On ELF the image then binds to the program's copy, so it loads only into an mcpp program that linked the package. A foreign host like CPython cannot load it, and Mach-O and PE refuse the link.

For a C library or a C++ runtime that is right: their contract decides where they go. It is not right for openkal-linux (-macos, -windows) on a hosted target. There the C library is the system's, and the kernel-abi package is an ordinary static library over it that mcpplibs.os depends on.

The fix

A mcpp:kernel-abi provider is a boundary only when a C library of the graph (mcpp:c-abi, e.g. openkal-musl) sits on it. Otherwise it is placed like any static package: into the one shared image that reaches it, or into the program when the program reaches it too.

Tests

  • New e2e test 940_a_kernel_abi_package_over_the_system_c_library_is_in_the_shared_image.sh. Its setup is a program over a dependency's shared image → static lib → fake kernel-abi package.
    • Leg A checks that the image defines the kernel-abi symbols. It fails on 2026.10.26.1 (U kal_fake_answer) and passes with the fix.
    • Leg B: a program that also reaches the package itself still links and runs.
  • Locally on Linux, these e2e tests pass: 08, 55, 56, 57, 222, 268, 280, 285, 286, 292, 307, 615, 704, 705, 740, 741, 743. mcpp test passes 151/151.
  • Downstream: openmui's Python extension (src/entry/python) imports and its CPython suite passes, with no hand-linking.

The changelog entry is left to the release PR, as usual.

…the shared image that reaches it

Every package providing an `mcpp:` layer was a boundary of static placement
(#646 F1), so openkal-linux/-macos/-windows went into the programs only. On a
hosted target that package is an ordinary static library over the platform's
C library (mcpplibs.os depends on it), and a shared image reaching it was left
with kal_* undefined: CPython importing an extension module built on
mcpplibs.os failed with `undefined symbol: kal_process_close`, and Mach-O and
PE refuse such a link. A kernel-abi provider is now a boundary only when a C
library of the graph (`mcpp:c-abi`) sits on it.

e2e 940: a program over a dependency's shared image; the image defines the
kernel-abi package's symbols (fails before the fix), and a program reaching
the package itself still links and runs.
@Sunrisepeak

Copy link
Copy Markdown
Member Author

The four openkal / build 3 targets failures are not from this change. Each fails while installing its toolchain, before anything is built:

error: xlings install_packages failed (exit 1) for 'openkal-musl@0.22.3' with an unknown index-repo configuration

The same jobs passed on release/2026.10.26.2 at 04:44Z, which resolved an earlier openkal-musl. They fail here since 06:48Z, and a rerun at 07:55Z failed the same way (macOS passed). That looks like the index entry for openkal-musl 0.22.3 rather than the planner. Every other check is green.

Sunrisepeak added a commit to Sunrisepeak/mui that referenced this pull request Oct 27, 2026
…example 03 in Python

- openmui.lang.python is openmui in Python, in host mode (D-46).
  - The extension is written against CPython's stable ABI (abi3,
    limited API 3.10) without CPython's headers, and links no
    libpython. The C API is resolved in the interpreter that loads it
    (mcpplibs.os 0.1.2's library), so one build serves every
    CPython 3.10+ of a platform.
  - Builders nest as `with` blocks. Operators on signals make
    expressions, and an expression has no truth value. Verbs are
    decorators; errors are OmuiError exceptions.
  - Every call releases the GIL and runs on openmui.lang's script
    thread. Handlers, verbs and watches take the GIL there; what they
    answer is not the runtime's.
- openmui.entry.python (src/entry/python) is the extension
  `_openmui`: the runtime on a virtual clock, the official components,
  and the desktop host adopting that runtime for a window.
  - It carries its own C++ runtime (cxx_runtime.shared =
    "self-contained").
  - settle() waits as the headless host does: the scripts idle,
    nothing animating, no timer due.
  - pixels() is the frame's FNV-1a, the hash the examples' tests pin.
- The wheel: lang/python/pyproject.toml with a PEP 517 backend in the
  tree (lang/python/packaging).
  - It builds the extension with mcpp and takes it from mcpp's result
    record.
  - It bundles the shared libraries beside it by the names the loader
    asks for, plus the third parties' notices.
  - It generates the typing stubs from the component catalogue
    (openmui._stubs; mypy --strict passes on example 03) and is tagged
    cp310-abi3-<platform>. Its archives are reproducible.
  - tools/ci/python-wheel.sh pip-installs it into a fresh venv. Against
    the installed package it runs the suite and example 03, and
    type-checks the example (M3 acceptance ⑤, Linux).
- Example 03 is written five times. python/tally.py has the same tree
  and pixels as the view document through every operation, and pins the
  C++ version's pixel hash.
- mcpp: a shared image now holds the kernel-abi package it reaches
  (openkal-linux over the system's C library). Without that, CPython
  refused the extension (`undefined symbol: kal_process_close`). Fixed
  in mcpp-community/mcpp#846, with e2e 940.
- The desktop host can adopt a runtime (DesktopOptions::runtime). The
  ABI wakes the host when work is posted (abi::set_wake), and
  element_from_json moved to openmui.lang for every binding.
- The workspace is on mcpplibs.os 0.1.2.
- Nightly installs esbuild, which examples 02 and 03 and lang/js need.
  It preloads the ASan runtime so the instrumented extension loads in
  CPython.
Sunrisepeak added a commit to Sunrisepeak/mui that referenced this pull request Oct 27, 2026
…community/mcpp#846)

tools/ci/python-extension-probe.sh builds openmui.entry.python and checks
that the library holds kal_* (nm on Linux and macOS; the link itself on
Windows). With an mcpp up to 2026.10.26.1 it does not: the extension, its
tests and the wheel step are left out of the run, with a warning naming the
mcpp fix; with a released fix they run again, with no change here. CI and
Nightly both probe.
Sunrisepeak added a commit to Sunrisepeak/mui that referenced this pull request Oct 27, 2026
…uilt after it

With a released mcpp, building openmui.entry.python before the workspace
left the GCC jobs' next link without kal_* (tests/bench). The workspace is now
built and tested without the extension first. Then, at the end of the job,
the probe builds the extension, and only if this mcpp links it do its suite
and the wheel step run (mcpp-community/mcpp#846). Nightly does the same.
@Sunrisepeak

Copy link
Copy Markdown
Member Author

Folded into the release PR #847 (commit a336ef0 on release/2026.10.27.1, the e2e renumbered 940 → 950 because 940 is taken on main; CHANGELOG entry under 2026.10.27.1 Fixed). This PR closes when #847 merges.

Co-authored-by: speak-agent 248744407+speak-agent@users.noreply.github.com

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant