feat(compile): auto-provision native ext bindings + wasm host - #7118
Open
jdalton wants to merge 1 commit into
Open
feat(compile): auto-provision native ext bindings + wasm host#7118jdalton wants to merge 1 commit into
jdalton wants to merge 1 commit into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (15)
💤 Files with no reviewable changes (1)
🚧 Files skipped from review as they are similar to previous changes (14)
📝 WalkthroughWalkthrough
ChangesAutomatic binding and Wasm provisioning
Estimated code review effort: 3 (Moderate) | ~30 minutes Sequence Diagram(s)sequenceDiagram
participant FFI as record_ffi_call
participant Registry as EXT_PREFIX_REGISTRY
participant Compile as perry compile
participant Resolver as optimized library resolver
participant Builder as build_wasm_host_library
participant Cargo
FFI->>Registry: Match extension symbol prefix
Registry-->>FFI: Return well-known binding provider
Compile->>Resolver: Resolve required static library
Resolver->>Builder: Request missing Wasm host archive
Builder->>Cargo: Build release archive for resolved target
Cargo-->>Builder: Produce static archive
Builder-->>Resolver: Return resolved archive path
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/perry/src/commands/compile/optimized_libs/driver.rs`:
- Around line 207-214: Update the missing-library auto-build call in the
optimized compilation flow to apply the target-specific Android or HarmonyOS
cross-compilation environment before invoking build_missing_prebuilt_ext_lib.
Reuse the same cross-env setup or target-specific invocation path used by
build_optimized_libs, while preserving existing behavior for other targets.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 898d2929-50d0-47ce-9118-6197d7efe94a
📒 Files selected for processing (7)
changelog.d/0000-auto-provision-ext-bindings.mdcrates/perry-codegen/src/ext_registry.rscrates/perry/src/commands/compile.rscrates/perry/src/commands/compile/library_search.rscrates/perry/src/commands/compile/optimized_libs/driver.rscrates/perry/src/commands/compile/optimized_libs/tests.rscrates/perry/src/commands/compile/run_pipeline.rs
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/perry/src/commands/compile/optimized_libs/no_auto.rs`:
- Around line 179-212: Update the HarmonyOS branch in the auto-build flow to
require both a usable clang compiler and the corresponding sysroot before
applying harmonyos_cross_env. Validate the candidate returned by
find_harmonyos_sdk, and when either requirement is missing, return None so the
existing fallback message/path is used instead of configuring an incomplete
cross environment.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: b3ba8319-22c9-4cf2-b3ab-916013f9b16e
📒 Files selected for processing (5)
crates/perry-runtime/src/os_network.rscrates/perry/src/commands/compile/optimized_libs/no_auto.rscrates/perry/src/commands/compile/resolve.rscrates/perry/src/commands/compile/well_known.rsworkspace-architecture.json
Drive native-artifact provisioning off what a program actually references instead of the well-known iteration set alone. Link: codegen's ext_registry gains a prefix net so any emitted js_<binding>_* FFI (js_ioredis_*, js_undici_*, js_node_forge_*) flips its well-known wrapper onto the link line off provenance alone. An AOT-compiled iovalkey/undici/node-forge (a compilePackages member, never in any import set) now links its perry-ext-* staticlib instead of failing with Undefined symbols: _js_ioredis_new. Build: routed CPU-only ext staticlibs are auto-built from workspace source when missing (reusing build_missing_prebuilt_ext_lib), and perry-wasm-host is auto-built when the program uses WebAssembly.* so libperry_wasm_host.a-not-found can't happen on a normal compile. Tests: ext_registry prefix routing + replay markers, ext binding key resolution + build-routing split, and a fake-cargo wasm-host auto-build. Review + CI follow-ups folded in: - Android/HarmonyOS cross-env: build_missing_prebuilt_ext_lib now applies the OHOS SDK / Android NDK compile env (mirroring build_optimized_libs) before the auto-build, so a C-dependent CPU-only wrapper's cross build no longer fails in build.rs before it can fall back (CodeRabbit). - workspace-architecture baseline refreshed to record the audit facts the ext crates imply: perry-ext-node-forge classification, the intentional perry-ext-node-forge/perry-ext-undici -> perry-runtime edges (PerryTS#6303/PerryTS#6314), perry-container-compose default membership, and member/closure counts. - Cleared pre-existing -D warnings dead-code that had been red on main since PerryTS#7080/PerryTS#7028/PerryTS#7031: gate perry-runtime's format_mac_address to its real callers (Windows/macOS/BSD/test; Linux reads the MAC preformatted from /sys), and allow(dead_code) on resolve_exports and the well_known provenance fields (read by the review gate + tests, not the link path). - Regenerated docs/src/api/reference.md + docs/api/perry.d.ts for the node-forge manifest entries added in PerryTS#7033. - Dropped the now-passing test_gap_zlib_3285_params from the gap snapshot.
jdalton
force-pushed
the
fix/auto-provision-ext-bindings
branch
from
July 31, 2026 23:24
05d3a38 to
410ffd3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A bare
perry compilefailed to fully provision the native artifacts a program actually needs. Two measured symptoms (Socket Firewall, perry v0.5.1265):iovalkeyinperry.compilePackages, so it AOT-compiles — but codegen still lowers its client usage to the nativejs_ioredis_newFFI. That symbol was in neither the exactext_registrytable nor any import set, so the well-known flip never fired andperry-ext-iorediswas never linked:Undefined symbols: _js_ioredis_new. This bites any well-known-backed package that also lives incompilePackages(or whose ext FFI is codegen-emitted without the module being in the well-known iteration set).perry-ext-node-forge) norperry-wasm-host, so a human had tocargo build --release -p …them first, and aWebAssembly.*program died withlibperry_wasm_host.a not found.Root cause
Native-artifact provisioning was driven off the well-known iteration set (
ctx.native_module_imports+PERRY_FORCE_WELL_KNOWN), not off what the program actually references.compilePackagesmembers are AOT-compiled, so they never enternative_module_imports— yet their codegen still emits the ext binding'sjs_<binding>_*FFI. The exact-symbolext_registrytable only routed symbols someone remembered to add (http/net/ws/events/mysql2/streams/crypto); wholejs_<binding>_*families (ioredis, undici, node-forge) fell through. Separately, only the shared-tokio ext crates were rebuilt in the auto-optimize cargo invocation; CPU-only ext crates andperry-wasm-hostwere built by nothing.Fix
Drive linking + building off the emitted-FFI set (the ground truth) plus the existing well-known routing.
crates/perry-codegen/src/ext_registry.rs) — add a prefix net (EXT_PREFIX_REGISTRY) that routes any emittedjs_<binding>_*symbol to its well-known binding key (js_ioredis_*→ioredis,js_undici_*→undici,js_node_forge_*→node-forge), checked after the exact table + crypto net. The driver's existingWellKnown(key)→native_module_imports→ routing machinery does the rest — no per-symbol maintenance, no import required. The per-module capture records the matched prefix as a stable object-cache replay marker (same shape as thejs_crypto_marker).optimized_libs/driver.rs) — when a routed CPU-only binding's staticlib is missing, auto-build it from workspace source via the existingbuild_missing_prebuilt_ext_libleaf-build (the shared-tokio ones are already rebuilt in the auto-optimize invocation). No-op when the archive is already present.library_search.rs+run_pipeline.rs) — addbuild_wasm_host_library, wired to the existingctx.needs_wasm_runtime/--enable-wasm-runtimesignal: when aWebAssembly.*program'slibperry_wasm_host.ais missing, buildperry-wasm-hostfrom workspace source before the symbol-stub scan + link so both resolve it. No-op when it exists or the program doesn't use wasm.Acceptance (bare
perry compile, noPERRY_FORCE_WELL_KNOWN, no manual prebuild)All three sfw entrypoints link and run;
_js_ioredis_newis gone;ioredis/http/netauto-build:src/sfw-enterprise/cli.ts--versionok)src/sfw-free/cli.ts--versionok)src/sfw-registry/cli.tsWasm host auto-provision verified end-to-end on a minimal
new WebAssembly.Module(...)program: withlibperry_wasm_host.aabsent, the compile auto-builds it (Using wasmi WebAssembly host runtime), links, and runs — thenot founderror is gone.Tests
perry-codegen: prefix routing for the ioredis/undici/node-forge families, no-over-match, and prefix-marker cache-replay.perry: ext binding keys resolve to theirperry-ext-*crates, the tokio vs CPU-only build-routing split, and a fake-cargoperry-wasm-hostauto-build.cargo test -p perry --bin perrygreen (841 passed);cargo build --release -p perrysucceeds;cargo fmtclean; changelog fragment added.Summary by CodeRabbit
perry compileautomatically detects, builds, and links required native extension bindings.node-forgemodule andtls.convertALPNProtocols.