diff --git a/CHANGELOG.md b/CHANGELOG.md index d32bfef13..557c5e105 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,7 @@ All notable changes to scriptc will be documented in this file. ### Features +- **Foreign-thread native callbacks are marshalled to the event loop.** FFI format 5 adds `invoke: "foreign"` for retained, context-bearing, `void` callbacks. Thread-safe generated trampolines copy scalar/string/byte arguments into plain staging memory, wake the process loop, and return immediately; the loop delivers one callback per turn on the script thread with ref'd registration liveness, concurrent-producer FIFO safety, explicit release, throw propagation, and clean shutdown across both backends. - **Native callbacks can be retained and explicitly released.** FFI format 4 adds `lifetime: "retained"` registrations and paired `release` descriptors that reuse the original function-pointer trampoline. Registrations pin captured closures until the same function value is released, count duplicate registrations, support multiple context-bearing registrations and raw single-slot replacement, defer callback throws through later FFI pump calls, and clean up live registrations at process exit across both backends. - **Native callbacks copy in strings and byte spans.** FFI format 3 adds callback-only `cstring` parameters plus length-delimited `string` and `bytes` parameters. Trampolines in both backends copy native memory into owned scriptc values, decode malformed UTF-8 with U+FFFD replacement, preserve embedded NUL bytes in spans, and trap precise invalid null pointers before invoking the closure. diff --git a/docs/src/app/ffi/page.mdx b/docs/src/app/ffi/page.mdx index ad224036a..1b4effd14 100644 --- a/docs/src/app/ffi/page.mdx +++ b/docs/src/app/ffi/page.mdx @@ -189,7 +189,7 @@ The callback id connects the two independently positioned context entries. Both } ``` -A callback descriptor consumes one TypeScript function parameter and one native function-pointer slot. A context entry consumes no TypeScript parameter and one native `void *` slot. Formats 2 through 4 accept `f64`, `bool`, `u8`, `u32`, and `i32` callback parameters plus at most one context entry. Formats 3 and 4 additionally accept `cstring`, `string`, and `bytes`; callback returns remain scalar or `void`. +A callback descriptor consumes one TypeScript function parameter and one native function-pointer slot. A context entry consumes no TypeScript parameter and one native `void *` slot. Formats 2 through 5 accept `f64`, `bool`, `u8`, `u32`, and `i32` callback parameters plus at most one context entry. Formats 3 through 5 additionally accept `cstring`, `string`, and `bytes`; callback returns remain scalar or `void`. Format 3 string-bearing callback parameters copy native data before the closure runs. `cstring` reads one non-null, NUL-terminated `const char *`. `string` and `bytes` each consume a `const uint8_t *, size_t` pair; a null pointer is valid only when its length is zero. Text is decoded as UTF-8 with malformed sequences replaced by U+FFFD, matching `Buffer.toString("utf8")`. The resulting string or `Uint8Array` is freshly owned scriptc storage, so the closure may retain it without depending on the native buffer's lifetime. An unexpected null `cstring`, or a null non-empty span, traps at the boundary instead of being treated as empty. @@ -243,19 +243,49 @@ timerRemove(tick); The release argument must be the same function value used for registration. Registrations to a context-bearing descriptor are counted: registering the same closure twice requires two releases. A raw descriptor's slot has replace semantics instead — every set call supersedes the previous registration, including one that passes the already-registered closure, so exactly one release is ever pending for that descriptor. Releasing an unregistered value traps because native code may still hold the original pointer — the trap fires before the native release call runs, so native code never observes the invalid release. The callback's function type must be exact at retained call sites; an implicit wrapper would create a different pointer and make release identity unsound. An inline function literal as a release argument is rejected for the same reason: it creates a fresh closure at every evaluation, a pointer no registration holds — pass the same named value used to register. Registering an inline literal remains legal; such a registration is simply permanent and is dropped by the exit teardown. A single binding cannot both register and release the same descriptor — the manifest loader rejects a `release` targeting a retained callback declared in the same function's parameter list, because the register-then-release ordering within one call would defeat the pre-call release validation. -Context-bearing descriptors support multiple concurrent closures. A raw retained descriptor has no context pointer, so it has one process-global slot with replace semantics: the previous registration stays live and dispatching until the replacing set call returns (a native setter that flushes the outgoing callback mid-replace still reaches the old closure), then it is released and the slot commits to the new closure. Retained registrations do not keep the event loop alive. At process exit, `process` `'exit'` listeners run first — they may still release or pump registrations on every exit path. On exits that run atexit handlers, the runtime then drops the remaining registrations and disarms raw slots; `process.exit()` terminates immediately after its listeners and skips that sweep, leaving remaining registrations to the operating system. A raw-slot invocation after teardown traps instead of reaching a freed closure. A context-bearing registration has no slot to disarm — its trampoline and context pointer dangle once teardown frees the closure — so native code must not invoke one after exit; a library that can fire on its own exit path should have its registrations released from a `process` `'exit'` listener. +Context-bearing descriptors support multiple concurrent closures. A raw retained descriptor has no context pointer, so it has one process-global slot with replace semantics: the previous registration stays live and dispatching until the replacing set call returns (a native setter that flushes the outgoing callback mid-replace still reaches the old closure), then it is released and the slot commits to the new closure. Script-thread retained registrations do not keep the event loop alive; format 5 foreign registrations do. At process exit, `process` `'exit'` listeners run first — they may still release registrations or pump script-thread callbacks on every exit path. Foreign posting is disarmed when the loop stops, so a straggling native post is silently dropped. On exits that run atexit handlers, the runtime then drops the remaining registrations and disarms raw slots; `process.exit()` terminates immediately after its listeners and skips that sweep, leaving remaining registrations to the operating system. A raw-slot invocation after teardown traps instead of reaching a freed closure. A context-bearing registration has no slot to disarm — its trampoline and context pointer dangle once teardown frees the closure — so native code must not invoke one after exit; a library that can fire on its own exit path should have its registrations released from a `process` `'exit'` listener. Retained identity is scoped to the declaring binding. Every retained callback parameter is its own descriptor: the `:` pair names one registration ledger, one generated trampoline, and (for a raw descriptor) one slot, and a release binding validates and unpins only registrations made through the binding its `release` reference targets. Two bindings that store into the same native state — a plain setter and a flush-on-replace setter for one native slot, say — are therefore independent descriptors that pass native code two different function pointers. Registering the same function value through both and then releasing it through one is unsound: the release unpins in its own descriptor's ledger, but native code compares stored pointers against the other descriptor's trampoline, so the surviving registration stays armed and keeps dispatching — nothing traps, and the callback keeps firing after the program believes it released it. Keep a function value registered with one such native registration point through exactly one binding at a time, and release it through that binding's paired release. -Both `"call"` and `"retained"` callbacks must run synchronously on the script thread. Direct foreign-thread invocation, value-returning foreign-thread callbacks, and real-time callback delivery are unsupported: scriptc's reference counting and exception cell are thread-confined, and a future foreign-thread surface must enqueue fire-and-forget delivery onto the script loop. Dereferenceable struct callback parameters are also unsupported; use an opaque native handle with accessor functions when the API permits it. +Format 5 adds `invoke: "foreign"` to a retained, context-bearing callback descriptor for libraries that invoke the callback from their own threads: -If a callback throws, the adapter returns zero (or `void`) to native code and suppresses further script callback execution while the exception is pending. When the outer native function returns, the original exception resumes through scriptc's ordinary catchable unwind path. Native work performed between the callback's return and the outer function's return is not rolled back. +```json:ffi.json +{ + "ffi_format": 5, + "functions": [ + { + "name": "timerAdd", + "symbol": "timer_add", + "params": [ + "u32", + { + "callback": { + "id": "tick", + "params": ["cstring", { "context": "tick" }], + "returns": "void", + "lifetime": "retained", + "invoke": "foreign" + } + }, + { "context": "tick" } + ], + "returns": "void" + } + ] +} +``` + +The native trampoline never runs script code. It copies scalar values and native string/byte memory into plain staging storage, posts to the process event loop, and returns immediately—even if native code happened to invoke it on the script thread. The loop delivers one invocation per turn, FIFO by enqueue order, with microtasks and timers interleaved. Live foreign registrations are ref'd: they keep the loop alive until their paired release binding runs. A callback may release itself; already-enqueued deliveries remain valid and drain before its closure pin is dropped. Throws follow ordinary timer-callback behavior and are uncaught unless the surrounding loop-dispatch semantics catch them. + +Foreign delivery is deliberately fire-and-forget. It requires `lifetime: "retained"`, `returns: "void"`, and a context entry. Value-returning foreign callbacks would have to block the library thread on the script loop and are refused as deadlock-prone. Delivery takes at least one loop turn and is not suitable for real-time work such as audio DSP. Direct execution of script closures on native threads remains permanently unsupported because reference counting, exception cells, and fibers are thread-confined. Dereferenceable struct callback parameters are also unsupported; use an opaque native handle with accessor functions when the API permits it. + +If a script-thread callback throws, the adapter returns zero (or `void`) to native code and suppresses further script callback execution while the exception is pending. When the outer native function returns, the original exception resumes through scriptc's ordinary catchable unwind path. Native work performed between the callback's return and the outer function's return is not rolled back. A foreign callback's native trampoline has already returned before its closure runs; a throw therefore follows event-loop callback semantics instead. ## Manifest fields
ffi_format
-
Required. Format 1 supports value parameters; format 2 preserves them and adds callback/context entries; format 3 adds copy-in cstring, string-span, and byte-span callback parameters; format 4 adds retained registrations and release references.
+
Required. Format 1 supports value parameters; format 2 preserves them and adds callback/context entries; format 3 adds copy-in cstring, string-span, and byte-span callback parameters; format 4 adds retained registrations and release references; format 5 adds retained foreign-thread callbacks marshalled to the event loop.
functions
Required array. Every entry has exactly name, symbol, params, and returns. Binding names and symbols must be unique. Callback ids must be unique within a function and every context must match exactly one callback or release. A release references a retained <binding>:<callback-id> in the same manifest and inherits its callback ABI.
@@ -273,7 +303,7 @@ Unknown fields, invalid ABI classes, duplicate names, and signature mismatches f - Native calls are synchronous and must return normally. Do not unwind C++ exceptions or `longjmp` across the boundary. - Native code is outside scriptc's exception, reference-counting, and sanitizer contracts. A bad pointer or mismatched C signature can still corrupt the process. -- Callbacks are same-thread only. Format 4 supports retained callbacks with explicit release; foreign-thread invocation is not supported. +- `invoke` defaults to `"script-thread"`. Format 5 foreign callbacks are asynchronous, `void`-returning, context-bearing, explicitly released, and not real-time capable; direct foreign-thread script execution is unsupported. - There are no variadic calls, struct-by-value arguments, owned pointer returns, or runtime `dlopen`/`dlsym` handles yet. - The archive or object must match the build target. Cross-compilation does not translate native inputs. - Outbound FFI is currently available for executable builds, not `scriptc build --lib`. diff --git a/docs/src/app/limitations/page.mdx b/docs/src/app/limitations/page.mdx index 00e230ea3..8307f8b3d 100644 --- a/docs/src/app/limitations/page.mdx +++ b/docs/src/app/limitations/page.mdx @@ -71,6 +71,8 @@ const who = process.argv.length > 2 ? process.argv[2] : "world"; **Memory is reference-counted.** Acyclic values free deterministically; reference cycles are collected at deterministic collection points, not by a concurrent GC. Cycles that cross the static/island boundary are uncollectable by either side. +**Foreign native callbacks are asynchronous and not real-time capable.** FFI format 5 accepts retained, context-bearing, `void` callbacks invoked by library-owned threads, but the native trampoline only copies arguments and enqueues work. The closure runs on the script event loop at least one turn later. Value-returning foreign callbacks and direct execution on the library thread are refused: waiting for the loop is deadlock-prone, while scriptc's reference counting and exception state are thread-confined. + **Process shape** — `process.argv[0]` is `"scriptc"` and `argv[1]` is the binary's path (positions line up with Node; `argv[2]` onward are your args). The uncaught-exception stderr line reads `Uncaught ` instead of Node's stack-trace block (exit code and pre-throw stdout are identical). Runtime errors carry `message` and Node's `code`, but not `errno`/`syscall`/`path`. **Comparator call sequences differ in `sort` and `toSorted`** (stable insertion sort here, TimSort in V8 — sorted results are byte-identical for consistent comparators), and **`localeCompare` compares code units**, not ICU collation. @@ -90,5 +92,5 @@ The production wasm32-wasi target supports the complete executable ## Tooling gaps - `scriptc run` does not forward extra CLI arguments to the program — `build` and invoke the binary directly. -- Native FFI is a direct, manifest-declared C ABI link surface. Callback invocation is same-thread only; format 4 supports explicitly released retained callbacks, but foreign-thread callbacks remain unsupported. Variadic calls, structs by value, owned pointer/string/byte returns, runtime dynamic-library loading, and library-mode builds also remain unsupported. See [Native FFI](/ffi). +- Native FFI is a direct, manifest-declared C ABI link surface. Formats 2–5 cover call-scoped callbacks, copied string/byte callback parameters, explicitly released retained callbacks, and asynchronous foreign-thread delivery. Variadic calls, structs by value, owned pointer/string/byte returns, runtime dynamic-library loading, and library-mode builds remain unsupported. See [Native FFI](/ffi). - Numbers are JS-exact f64 everywhere. Integer inference and ownership analysis — the systems-language performance ceiling — are roadmap, not shipped. diff --git a/packages/compiler/src/backend/cc.ts b/packages/compiler/src/backend/cc.ts index e4e3fa512..0ae232321 100644 --- a/packages/compiler/src/backend/cc.ts +++ b/packages/compiler/src/backend/cc.ts @@ -345,6 +345,10 @@ export interface CcOptions { * scr_watch.c into the binary — the net gating precedent, so watch-free * binaries keep their exact link line. */ watch?: boolean; + /** The executable manifest has a format-5 foreign callback descriptor: + * compiles the MPSC queue/self-pipe unit. Other FFI and non-FFI binaries + * keep their existing runtime size class. */ + foreignFfi?: boolean; /** The program uses node:test (moduleUsesNodeTest on the IR): compiles * scr_test.c into the binary — the net gating precedent, so test-free * binaries keep their exact link line. */ @@ -3638,6 +3642,7 @@ export async function compileC(opts: CcOptions): Promise { ...(opts.http2 ?? false ? [rt(join(rtDir, "scr_http2.c"))] : []), ...(opts.dgram ? [rt(join(rtDir, "scr_dgram.c"))] : []), ...(opts.watch ? [rt(join(rtDir, "scr_watch.c"))] : []), + ...(opts.foreignFfi ? [rt(join(rtDir, "scr_ffi_queue.c"))] : []), ...(opts.nodeTest ? [rt(join(rtDir, "scr_test.c"))] : []), // The CA-store unit rides its own gate OR the tls one: scr_tls.c // references its default-set override unconditionally. diff --git a/packages/compiler/src/backend/emission/emit-exprs.ts b/packages/compiler/src/backend/emission/emit-exprs.ts index 871f6c817..93fe905aa 100644 --- a/packages/compiler/src/backend/emission/emit-exprs.ts +++ b/packages/compiler/src/backend/emission/emit-exprs.ts @@ -2148,6 +2148,8 @@ export function emitExpr(E: CEmitter, e: IrExpr): Temp { for (const registration of retainedRegistrations) { if (registration.global !== null) { E.line(`scr_ffi_retain_slot(&${registration.table}, &${registration.global}, ${registration.callback.name});`); + } else if (registration.foreign) { + E.line(`scr_ffi_retain_foreign(&${registration.table}, ${registration.callback.name});`); } else { E.line(`scr_ffi_retain(&${registration.table}, ${registration.callback.name});`); } @@ -2156,7 +2158,7 @@ export function emitExpr(E: CEmitter, e: IrExpr): Temp { // release traps without native code observing any side effect. The // registration itself is unpinned only after the call returns. for (const release of retainedReleases) { - E.line(`scr_ffi_require(&${release.table}, ${release.callback.name});`); + E.line(`scr_ffi_require${release.foreign ? "_foreign" : ""}(&${release.table}, ${release.callback.name});`); } // Raw C callback pointers carry no userdata. For the documented @@ -2234,7 +2236,7 @@ export function emitExpr(E: CEmitter, e: IrExpr): Temp { } } for (const release of retainedReleases) { - E.line(`scr_ffi_release(&${release.table}, ${release.callback.name});`); + E.line(`scr_ffi_release${release.foreign ? "_foreign" : ""}(&${release.table}, ${release.callback.name});`); } }; const callbacksMayThrow = callbackArgs.size > 0 || E.ffiHasRetainedCallback; diff --git a/packages/compiler/src/backend/emission/emitter.ts b/packages/compiler/src/backend/emission/emitter.ts index 223153c69..34e99288b 100644 --- a/packages/compiler/src/backend/emission/emitter.ts +++ b/packages/compiler/src/backend/emission/emitter.ts @@ -40,7 +40,7 @@ import type { SrcLoc, } from "../../ir/nodes.js"; import { ffiCallbackType, funcOf, isFfiCallbackParam, isFfiContextParam, isFfiReleaseParam, isRefCounted, isUnitType, mapOf, moduleEmbedsCompressedNpm, moduleUsesDgram, moduleUsesDynInvoke, moduleEmbedsBuiltin, moduleUsesFetch, moduleUsesFsWatch, moduleUsesHttp2, moduleUsesHttpServer, moduleUsesNet, moduleUsesNodeTest, moduleUsesProcessEvents, moduleUsesStream, moduleUsesTls, moduleUsesTlsCa, RUNTIME_EMITTER_CLASS, STRING, VOID } from "../../ir/nodes.js"; -import { allocateFfiCallbackAdapters, hasRetainedFfiCallback, type FfiCallbackAdapter } from "../ffi-callbacks.js"; +import { allocateFfiCallbackAdapters, hasForeignFfiCallback, hasRetainedFfiCallback, type FfiCallbackAdapter } from "../ffi-callbacks.js"; import { mangleAsyncSpawn, mangleGenSpawn, @@ -247,6 +247,7 @@ export class CEmitter { * pending-exception checkpoint (may-throw derives the same fact from * the same helper). */ readonly ffiHasRetainedCallback: boolean; + readonly ffiHasForeignCallback: boolean; readonly globalsById = new Map(); readonly unionsById = new Map(); /** Active optional-chain bind temps, by chain id (chainRecv reads). */ @@ -413,6 +414,7 @@ export class CEmitter { ) { this.ffiCallbackAdapters = allocateFfiCallbackAdapters(mod.ffiImports ?? []); this.ffiHasRetainedCallback = hasRetainedFfiCallback(mod.ffiImports ?? []); + this.ffiHasForeignCallback = hasForeignFfiCallback(mod.ffiImports ?? []); for (const fn of mod.functions) { this.returnTypeByFn.set(fn.name, fn.returnType); this.fnByName.set(fn.name, fn); @@ -957,6 +959,7 @@ export class CEmitter { // fs.watch programs fill the loop's watch hooks the same way — // scr_watch.c links only when this line is emitted. ...(moduleUsesFsWatch(this.mod) ? [` scr_watch_install();`] : []), + ...(this.ffiHasForeignCallback ? [` scr_ffi_install();`] : []), // The embedded npm tables must be registered before %main: the %init // functions it calls import from them. Static data only — the engine // still boots lazily, on the first island entry. Compressed module @@ -981,7 +984,7 @@ export class CEmitter { // The event loop runs to exhaustion (microtasks before timers). A // throw escaping a timer callback and unhandled promise rejections // both exit 1, like Node. - ...(hasAsync || hasGenerators || this.usesTimers || usesIsland + ...(hasAsync || hasGenerators || this.usesTimers || usesIsland || this.ffiHasForeignCallback ? [ ` bool sc_loop_rejection = scr_loop_run(${asyncEntry ? "sc_top" : "NULL"});`, ...uncaught(" ", asyncEntry), @@ -1785,6 +1788,70 @@ export class CEmitter { const nativeParams = ffiCallbackNativeParamsC(cb, true); const ret = ffiNativeTypeC(cb.returns); const contextParam = cb.params.findIndex(isFfiContextParam); + if (cb.invoke === "foreign") { + if (adapter.table === null || contextParam < 0 || cb.returns !== "void") { + throw new Error("emitter bug: invalid foreign FFI callback descriptor"); + } + const dispatch = `${adapter.symbol}_dispatch`; + const ft = ffiCallbackType(cb); + const scriptArgs = cb.params.flatMap((param, i): string[] => { + if (isFfiContextParam(param)) return []; + switch (param) { + case "f64": + return [`scr_ffi_call_get_f64(sc_call, ${i})`]; + case "bool": + return [`scr_ffi_call_get_bool(sc_call, ${i})`]; + case "u8": + return [`scr_ffi_call_get_u8(sc_call, ${i})`]; + case "u32": + return [`scr_ffi_call_get_u32(sc_call, ${i})`]; + case "i32": + return [`scr_ffi_call_get_i32(sc_call, ${i})`]; + case "cstring": + case "string": + return [`scr_str_from_utf8_lossy(scr_ffi_call_get_data(sc_call, ${i}), scr_ffi_call_get_len(sc_call, ${i}))`]; + case "bytes": + return [`scr_bytes_from_data(scr_ffi_call_get_data(sc_call, ${i}), scr_ffi_call_get_len(sc_call, ${i}))`]; + } + }); + out.push( + `static void ${dispatch}(ScrClosure *sc_cb, ScrFfiCall *sc_call) {`, + ` (${cFnPtrCast(ft)}sc_cb->fn)(sc_cb${scriptArgs.length ? `, ${scriptArgs.join(", ")}` : ""});`, + `}`, + `static void ${adapter.symbol}(${nativeParams.join(", ")}) {`, + ` ScrClosure *sc_cb = (ScrClosure *)sc_ctx;`, + ` ScrFfiCall *sc_call = scr_ffi_call_new(&${adapter.table}, sc_cb, &${dispatch}, ${cb.params.length});`, + ); + cb.params.forEach((param, i) => { + if (isFfiContextParam(param)) return; + switch (param) { + case "f64": + out.push(` scr_ffi_call_set_f64(sc_call, ${i}, sc_a${i});`); + break; + case "bool": + out.push(` scr_ffi_call_set_bool(sc_call, ${i}, sc_a${i});`); + break; + case "u8": + out.push(` scr_ffi_call_set_u8(sc_call, ${i}, sc_a${i});`); + break; + case "u32": + out.push(` scr_ffi_call_set_u32(sc_call, ${i}, sc_a${i});`); + break; + case "i32": + out.push(` scr_ffi_call_set_i32(sc_call, ${i}, sc_a${i});`); + break; + case "cstring": + out.push(` scr_ffi_call_copy_cstring(sc_call, ${i}, sc_a${i});`); + break; + case "string": + case "bytes": + out.push(` scr_ffi_call_copy_${param}(sc_call, ${i}, sc_a${i}, sc_a${i}_len);`); + break; + } + }); + out.push(` scr_ffi_post(sc_call);`, `}`, ``); + continue; + } out.push( `static ${ret} ${adapter.symbol}(${nativeParams.length > 0 ? nativeParams.join(", ") : "void"}) {`, ` ScrClosure *sc_cb = ${contextParam >= 0 ? `(ScrClosure *)sc_ctx` : adapter.tls ?? adapter.global};`, diff --git a/packages/compiler/src/backend/ffi-callbacks.ts b/packages/compiler/src/backend/ffi-callbacks.ts index 1b9a90154..f3b988790 100644 --- a/packages/compiler/src/backend/ffi-callbacks.ts +++ b/packages/compiler/src/backend/ffi-callbacks.ts @@ -87,6 +87,16 @@ export function hasRetainedFfiCallback(imports: readonly IrFfiImport[]): boolean ); } +/** Foreign callbacks install and hold the process event loop even when the + * source itself has no timer, async function, or other loop-backed surface. */ +export function hasForeignFfiCallback(imports: readonly IrFfiImport[]): boolean { + return imports.some((entry) => + entry.params.some( + (param) => isFfiCallbackParam(param) && param.callback.invoke === "foreign", + ), + ); +} + /** One retained lifecycle operation of an FFI call: the registration table, * the raw singleton trampoline slot (null for context-bearing descriptors), * and the backend's value for the closure argument. */ @@ -94,6 +104,7 @@ export interface FfiRetainedOp { table: string; global: string | null; callback: V; + foreign: boolean; } /** Collect a call's retained registrations and releases in manifest order — @@ -114,6 +125,7 @@ export function collectFfiRetainedOps( table: adapter.table, global: adapter.global, callback: callbackArgs.get(param.callback.id)!, + foreign: param.callback.invoke === "foreign", }); } else if (isFfiReleaseParam(param)) { const { binding, id } = parseFfiCallbackKey(param.callback.release); @@ -123,6 +135,7 @@ export function collectFfiRetainedOps( table: adapter.table, global: adapter.global, callback: callbackArgs.get(param.callback.release)!, + foreign: adapter.callback.invoke === "foreign", }); } } diff --git a/packages/compiler/src/backend/llvm/emitter.ts b/packages/compiler/src/backend/llvm/emitter.ts index e4b24414a..b686d2ac7 100644 --- a/packages/compiler/src/backend/llvm/emitter.ts +++ b/packages/compiler/src/backend/llvm/emitter.ts @@ -81,7 +81,7 @@ import type { } from "../../ir/nodes.js"; import { canMarshalFuncIntoIsland, CAUGHT, DYN, F64, ffiCallbackType, islandCallbackRet, islandPromisePayloadTag, isFfiCallbackParam, isFfiContextParam, isFfiReleaseParam, isRefCounted, isUnitType, MAY_THROW_LIB_FNS, moduleEmbedsBuiltin, moduleEmbedsCompressedNpm, moduleUsesDynInvoke, moduleUsesFetch, moduleUsesFsWatch, moduleUsesHttpServer, moduleUsesNet, moduleUsesNodeTest, moduleUsesProcessEvents, moduleUsesStream, moduleUsesTls, moduleUsesTlsCa, NPM_COMPRESS_MIN, RUNTIME_EMITTER_CLASS, RUNTIME_ERROR_CLASSES, RUNTIME_STREAM_CLASSES, STRING, typeEquals, typeKey, VOID } from "../../ir/nodes.js"; import { matchIntegerBytesForLoop } from "../../ir/integer-loops.js"; -import { allocateFfiCallbackAdapters, collectFfiRetainedOps, hasRetainedFfiCallback, parseFfiCallbackKey, type FfiCallbackAdapter } from "../ffi-callbacks.js"; +import { allocateFfiCallbackAdapters, collectFfiRetainedOps, hasForeignFfiCallback, hasRetainedFfiCallback, parseFfiCallbackKey, type FfiCallbackAdapter } from "../ffi-callbacks.js"; import { computeMayThrow } from "../emission/may-throw.js"; import { mangleArgPack, mangleAsyncSpawn, mangleClassNew, mangleClassObj, mangleClassRetain, mangleFnClosure, mangleFunction, mangleGenDrop, mangleGenResThunk, mangleGenSpawn, mangleGlobal, mangleLocal, mangleRecordNew, mangleRecordStruct, mangleResolveThunk, mangleTrampoline, mangleVtStruct, mangleWrapper } from "../mangle.js"; import { BlockBuilder } from "./blocks.js"; @@ -1023,6 +1023,7 @@ class LlEmitter { * pending-exception checkpoint (may-throw derives the same fact from * the same helper). */ private readonly ffiHasRetainedCallback: boolean; + private readonly ffiHasForeignCallback: boolean; private readonly globalTypes = new Map(); /** May-throw analysis (the C emitter's computeMayThrow, shared): pending * checks are emitted only after calls that can actually raise. */ @@ -1145,6 +1146,7 @@ class LlEmitter { this.cycleColorOffset = options.pointerBits === 32 ? 12 : 16; this.ffiCallbackAdapters = allocateFfiCallbackAdapters(mod.ffiImports ?? []); this.ffiHasRetainedCallback = hasRetainedFfiCallback(mod.ffiImports ?? []); + this.ffiHasForeignCallback = hasForeignFfiCallback(mod.ffiImports ?? []); for (const fn of mod.functions) this.fnByName.set(fn.name, fn); for (const entry of mod.ffiImports ?? []) { this.ffiByName.set(entry.name, entry); @@ -1311,6 +1313,94 @@ class LlEmitter { return [`${ffiNativeTypeLl(param)} %a${i}`]; }); const ret = ffiNativeTypeLl(cb.returns); + if (cb.invoke === "foreign") { + if (adapter.table === null || !cb.params.some(isFfiContextParam) || cb.returns !== "void") { + throw new Error("llvm emitter bug: invalid foreign FFI callback descriptor"); + } + const dispatch = `${adapter.symbol}_dispatch`; + const scriptArgs: string[] = []; + const dispatchBody: string[] = [ + `define internal void @${dispatch}(ptr %cb, ptr %call) ${FN_ATTRS} {`, + `entry:`, + ` %fnp = getelementptr inbounds %ScrClosure, ptr %cb, i64 0, i32 1`, + ` %fn = load ptr, ptr %fnp`, + ]; + for (let i = 0; i < cb.params.length; i++) { + const param = cb.params[i]!; + if (isFfiContextParam(param)) continue; + switch (param) { + case "f64": + this.declare(`declare double @scr_ffi_call_get_f64(ptr, ${this.sizeType})`); + dispatchBody.push(` %s${i} = call double @scr_ffi_call_get_f64(ptr %call, ${this.sizeType} ${i})`); + scriptArgs.push(`double %s${i}`); + break; + case "bool": + this.declare(`declare zeroext i1 @scr_ffi_call_get_bool(ptr, ${this.sizeType})`); + dispatchBody.push(` %s${i} = call zeroext i1 @scr_ffi_call_get_bool(ptr %call, ${this.sizeType} ${i})`); + scriptArgs.push(`i1 %s${i}`); + break; + case "u8": + case "u32": + case "i32": + this.declare(`declare double @scr_ffi_call_get_${param}(ptr, ${this.sizeType})`); + dispatchBody.push(` %s${i} = call double @scr_ffi_call_get_${param}(ptr %call, ${this.sizeType} ${i})`); + scriptArgs.push(`double %s${i}`); + break; + case "cstring": + case "string": + case "bytes": { + this.declare(`declare ptr @scr_ffi_call_get_data(ptr, ${this.sizeType})`); + this.declare(`declare ${this.sizeType} @scr_ffi_call_get_len(ptr, ${this.sizeType})`); + dispatchBody.push( + ` %data${i} = call ptr @scr_ffi_call_get_data(ptr %call, ${this.sizeType} ${i})`, + ` %len${i} = call ${this.sizeType} @scr_ffi_call_get_len(ptr %call, ${this.sizeType} ${i})`, + ); + if (param === "bytes") { + this.declare(`declare ptr @scr_bytes_from_data(ptr, ${this.sizeType})`); + dispatchBody.push(` %s${i} = call ptr @scr_bytes_from_data(ptr %data${i}, ${this.sizeType} %len${i})`); + } else { + this.declare(`declare ptr @scr_str_from_utf8_lossy(ptr, ${this.sizeType})`); + dispatchBody.push(` %s${i} = call ptr @scr_str_from_utf8_lossy(ptr %data${i}, ${this.sizeType} %len${i})`); + } + scriptArgs.push(`ptr %s${i}`); + break; + } + } + } + dispatchBody.push( + ` call void %fn(${[`ptr %cb`, ...scriptArgs].join(", ")})`, + ` ret void`, + `}`, + ``, + ); + defs.push(...dispatchBody); + + this.declare(`declare ptr @scr_ffi_call_new(ptr, ptr, ptr, ${this.sizeType})`); + this.declare(`declare void @scr_ffi_post(ptr)`); + defs.push( + `define internal void @${adapter.symbol}(${params.join(", ")}) ${FN_ATTRS} {`, + `entry:`, + ` %cb = getelementptr inbounds i8, ptr %ctx, i64 0`, + ` %call = call ptr @scr_ffi_call_new(ptr @${adapter.table}, ptr %cb, ptr @${dispatch}, ${this.sizeType} ${cb.params.length})`, + ); + for (let i = 0; i < cb.params.length; i++) { + const param = cb.params[i]!; + if (isFfiContextParam(param)) continue; + if (param === "cstring") { + this.declare(`declare void @scr_ffi_call_copy_cstring(ptr, ${this.sizeType}, ptr)`); + defs.push(` call void @scr_ffi_call_copy_cstring(ptr %call, ${this.sizeType} ${i}, ptr %a${i})`); + } else if (param === "string" || param === "bytes") { + this.declare(`declare void @scr_ffi_call_copy_${param}(ptr, ${this.sizeType}, ptr, ${this.sizeType})`); + defs.push(` call void @scr_ffi_call_copy_${param}(ptr %call, ${this.sizeType} ${i}, ptr %a${i}, ${this.sizeType} %a${i}_len)`); + } else { + const nativeTy = ffiNativeTypeLl(param); + this.declare(`declare void @scr_ffi_call_set_${param}(ptr, ${this.sizeType}, ${nativeTy})`); + defs.push(` call void @scr_ffi_call_set_${param}(ptr %call, ${this.sizeType} ${i}, ${nativeTy} %a${i})`); + } + } + defs.push(` call void @scr_ffi_post(ptr %call)`, ` ret void`, `}`, ``); + continue; + } defs.push( `define internal ${ret} @${adapter.symbol}(${params.join(", ")}) ${FN_ATTRS} {`, `entry:`, @@ -1575,6 +1665,7 @@ class LlEmitter { // Declared NOW — the extern block flushes before main assembles. if (usesEvents) this.declare(`declare void @scr_events_install()`); if (usesFsWatch) this.declare(`declare void @scr_watch_install()`); + if (this.ffiHasForeignCallback) this.declare(`declare void @scr_ffi_install()`); if (usesStream) this.declare(`declare void @scr_stream_install()`); if (usesNet) { this.declare(`declare void @scr_net_install()`); @@ -1629,6 +1720,7 @@ class LlEmitter { const runsLoop = this.usesTimers || usesIsland || + this.ffiHasForeignCallback || this.mod.functions.some((f) => f.async === true || f.generator !== undefined); const uncaughtReleases = entryMayThrow && !asyncEntry ? globalReleaseLines("gu") : []; const loopReleasesU = runsLoop ? globalReleaseLines("gl") : []; @@ -1751,7 +1843,7 @@ class LlEmitter { `%ScrVt = type { ${this.sizeType}, ${this.sizeType}, ptr }`, `%ScrUnion = type { ${this.sizeType}, i32, ptr, ptr, ptr, i64 }`, `%ScrClosure = type { ${this.sizeType}, ptr, ${this.sizeType}, ptr }`, - `%ScrFfiTable = type { ptr, ${this.sizeType}, ${this.sizeType}, ptr, i8, ptr }`, + `%ScrFfiTable = type { ptr, ${this.sizeType}, ${this.sizeType}, ptr, i8, ptr, ptr, ${this.sizeType}, ${this.sizeType}, ${this.sizeType}, ptr, ptr }`, `%ScrRegex = type { ${this.sizeType}, ptr, ptr, ptr }`, // ScrArr mirror { rc, len, cap, elem(i32+pad), elem_retain, // elem_release, elem_trace, data } — the immortal tagged-template @@ -1991,6 +2083,7 @@ class LlEmitter { // fs.watch programs fill the loop's watch hooks the same way — // scr_watch.c links only when this line is emitted. ...(usesFsWatch ? [` call void @scr_watch_install()`] : []), + ...(this.ffiHasForeignCallback ? [` call void @scr_ffi_install()`] : []), ...(snapshotsTlsCa ? [` call void @scr_tls_ca_install()`] : []), ...(usesFetch ? [` call void @scr_fetch_install()`] : []), ...(embedsZlib ? [` call void @scr_zlib_island_install()`] : []), @@ -6062,6 +6155,9 @@ class LlEmitter { } if (retainedReleases.length > 0) { this.declare(`declare void @scr_ffi_require(ptr, ptr)`); + if (retainedReleases.some((release) => release.foreign)) { + this.declare(`declare void @scr_ffi_require_foreign(ptr, ptr)`); + } } // Pin before registration. Raw retained descriptors are native // singletons: the incoming closure is pinned (and an EMPTY slot @@ -6073,6 +6169,9 @@ class LlEmitter { for (const registration of retainedRegistrations) { if (registration.global !== null) { B.line(`call void @scr_ffi_retain_slot(ptr @${registration.table}, ptr @${registration.global}, ptr ${registration.callback.name})`); + } else if (registration.foreign) { + this.declare(`declare void @scr_ffi_retain_foreign(ptr, ptr)`); + B.line(`call void @scr_ffi_retain_foreign(ptr @${registration.table}, ptr ${registration.callback.name})`); } else { B.line(`call void @scr_ffi_retain(ptr @${registration.table}, ptr ${registration.callback.name})`); } @@ -6081,7 +6180,7 @@ class LlEmitter { // release traps without native code observing any side effect. The // registration itself is unpinned only after the call returns. for (const release of retainedReleases) { - B.line(`call void @scr_ffi_require(ptr @${release.table}, ptr ${release.callback.name})`); + B.line(`call void @scr_ffi_require${release.foreign ? "_foreign" : ""}(ptr @${release.table}, ptr ${release.callback.name})`); } const rawContexts: { tls: string; previous: string }[] = []; @@ -6205,9 +6304,14 @@ class LlEmitter { B.line(`call void @scr_ffi_commit_slot(ptr @${registration.table}, ptr ${registration.callback.name})`); } } - if (retainedReleases.length > 0) this.declare(`declare void @scr_ffi_release(ptr, ptr)`); + if (retainedReleases.some((release) => !release.foreign)) { + this.declare(`declare void @scr_ffi_release(ptr, ptr)`); + } + if (retainedReleases.some((release) => release.foreign)) { + this.declare(`declare void @scr_ffi_release_foreign(ptr, ptr)`); + } for (const release of retainedReleases) { - B.line(`call void @scr_ffi_release(ptr @${release.table}, ptr ${release.callback.name})`); + B.line(`call void @scr_ffi_release${release.foreign ? "_foreign" : ""}(ptr @${release.table}, ptr ${release.callback.name})`); } }; const callbacksMayThrow = callbackArgs.size > 0 || this.ffiHasRetainedCallback; diff --git a/packages/compiler/src/ffi/profile.ts b/packages/compiler/src/ffi/profile.ts index daf73a1ff..9bb38f94c 100644 --- a/packages/compiler/src/ffi/profile.ts +++ b/packages/compiler/src/ffi/profile.ts @@ -51,7 +51,15 @@ * JSON: * * { "callback": { "release": "timerAdd:tick" } } - * { "context": "timerAdd:tick" } */ + * { "context": "timerAdd:tick" } + * + * Format 5 adds retained foreign-thread callbacks. Their native trampoline + * only stages plain data and posts it to the process event loop; script code + * always runs asynchronously on the script thread: + * + * { "callback": { "id": "tick", "params": [{ "context": "tick" }], + * "returns": "void", "lifetime": "retained", + * "invoke": "foreign" } } */ import { readFileSync, statSync } from "node:fs"; import { dirname, resolve } from "node:path"; import { ffiProfileDiag, type ScrDiagnostic } from "../diagnostics/diagnostic.js"; @@ -97,6 +105,8 @@ export interface FfiCallbackParam { returns: FfiReturnClass; /** Dynamic-extent borrow, or explicitly paired retained registration. */ lifetime: "call" | "retained"; + /** Native invocation thread; foreign delivery is marshalled to the loop. */ + invoke: "script-thread" | "foreign"; }; } @@ -127,7 +137,7 @@ export interface FfiFunction { } export interface FfiProfile { - ffiFormat: 1 | 2 | 3 | 4; + ffiFormat: 1 | 2 | 3 | 4 | 5; functions: FfiFunction[]; /** Absolute paths, resolved relative to the manifest. */ libraries: string[]; @@ -188,7 +198,7 @@ type UnresolvedFfiParamClass = FfiParamClass | UnresolvedFfiReleaseParam; function callbackParam( value: unknown, path: string, - format: 2 | 3 | 4, + format: 2 | 3 | 4 | 5, ): FfiCallbackParam | UnresolvedFfiReleaseParam | null { if (value === null || typeof value !== "object" || Array.isArray(value)) return null; const rec = value as Record; @@ -208,7 +218,7 @@ function callbackParam( callback: { release: stringField(callback["release"], `${path}.callback.release`) }, }; } - rejectUnknownKeys(callback, `${path}.callback`, ["id", "params", "returns", "lifetime"]); + rejectUnknownKeys(callback, `${path}.callback`, ["id", "params", "returns", "lifetime", "invoke"]); const id = stringField(callback["id"], `${path}.callback.id`); if (!TS_IDENT.test(id)) { throw new FfiProfileError(`'${path}.callback.id' is not a plain identifier: '${id}'`); @@ -252,12 +262,29 @@ function callbackParam( if (lifetime === "retained" && format < 4) { throw new FfiProfileError(`'${path}.callback.lifetime' value 'retained' requires ffi_format 4`); } + const invoke = callback["invoke"] ?? "script-thread"; + if (invoke !== "script-thread" && invoke !== "foreign") { + throw new FfiProfileError(`'${path}.callback.invoke' must be 'script-thread' or 'foreign'`); + } + if (invoke === "foreign" && format < 5) { + throw new FfiProfileError(`'${path}.callback.invoke' value 'foreign' requires ffi_format 5`); + } + if (invoke === "foreign" && lifetime !== "retained") { + throw new FfiProfileError(`'${path}.callback.invoke' value 'foreign' requires lifetime 'retained'`); + } + if (invoke === "foreign" && returns !== "void") { + throw new FfiProfileError(`'${path}.callback.invoke' value 'foreign' requires returns 'void'`); + } + if (invoke === "foreign" && !params.some((param) => typeof param === "object")) { + throw new FfiProfileError(`'${path}.callback.invoke' value 'foreign' requires a context entry`); + } return { callback: { id, params, returns: returns as FfiReturnClass, lifetime, + invoke, }, }; } @@ -288,11 +315,11 @@ export function loadFfiProfile( } const root = raw as Record; const format = root["ffi_format"]; - if (format !== 1 && format !== 2 && format !== 3 && format !== 4) { + if (format !== 1 && format !== 2 && format !== 3 && format !== 4 && format !== 5) { throw new FfiProfileError( typeof format === "number" - ? `unsupported ffi_format ${format} (this scriptc reads formats 1, 2, 3, and 4)` - : "'ffi_format' must be the number 1, 2, 3, or 4", + ? `unsupported ffi_format ${format} (this scriptc reads formats 1, 2, 3, 4, and 5)` + : "'ffi_format' must be the number 1, 2, 3, 4, or 5", ); } rejectUnknownKeys(root, "", [ @@ -343,7 +370,7 @@ export function loadFfiProfile( !(FFI_PARAM_CLASSES as readonly string[]).includes(value) ) { if (format >= 2) { - const callback = callbackParam(value, paramPath, format as 2 | 3 | 4); + const callback = callbackParam(value, paramPath, format as 2 | 3 | 4 | 5); if (callback !== null) return callback; const context = contextParam(value, paramPath); if (context !== null) return context; diff --git a/packages/compiler/src/frontend/lowering/lowerer.ts b/packages/compiler/src/frontend/lowering/lowerer.ts index db04882d3..5cc456b5f 100644 --- a/packages/compiler/src/frontend/lowering/lowerer.ts +++ b/packages/compiler/src/frontend/lowering/lowerer.ts @@ -2265,7 +2265,7 @@ export class Lowerer { this.diags.length > 0 ? null : { - irVersion: 4, + irVersion: 5, sourceFile: this.entry.fileName, functions, classes: artifacts.classes, diff --git a/packages/compiler/src/index.ts b/packages/compiler/src/index.ts index 5f614973c..1b55371a6 100644 --- a/packages/compiler/src/index.ts +++ b/packages/compiler/src/index.ts @@ -32,6 +32,7 @@ import { isJsSourceFileName, isRelativeSpecifier } from "./frontend/shared.js"; import { lowerToIr, type LowerOptions, type LowerResult } from "./frontend/lowering/lowerer.js"; import type { CoverageInput, NpmStaticStatus } from "./coverage/report.js"; import { loadFfiProfile, type FfiProfile } from "./ffi/profile.js"; +import { hasForeignFfiCallback } from "./backend/ffi-callbacks.js"; export const VERSION = "0.0.1"; @@ -1055,6 +1056,7 @@ export async function compile(entryPath: string, opts: CompileOptions): Promise< dgram: moduleUsesDgram(lowered.module), // The link switch for scr_watch.c: fs.watch/watcher.* libCalls on the IR. watch: moduleUsesFsWatch(lowered.module), + foreignFfi: hasForeignFfiCallback(lowered.module.ffiImports ?? []), // The link switch for scr_test.c: test.* libCalls on the IR. nodeTest: moduleUsesNodeTest(lowered.module), // The link switch for scr_tls.c + the vendored mbedTLS archive: diff --git a/packages/compiler/src/ir/nodes.ts b/packages/compiler/src/ir/nodes.ts index c3471f36b..32ad7d469 100644 --- a/packages/compiler/src/ir/nodes.ts +++ b/packages/compiler/src/ir/nodes.ts @@ -693,7 +693,7 @@ export function isRefCounted(t: IrType): boolean { export interface IrModule { /** Bumped on any breaking IR change; serialize.ts refuses mismatches. */ - irVersion: 4; + irVersion: 5; sourceFile: string; functions: IrFunction[]; /** Class shapes. Constructors and methods are ordinary module functions @@ -786,6 +786,7 @@ export interface IrFfiCallbackParam { params: (IrFfiCallbackParamClass | IrFfiContextParam)[]; returns: IrFfiReturnClass; lifetime: "call" | "retained"; + invoke: "script-thread" | "foreign"; }; } diff --git a/packages/compiler/src/ir/serialize.ts b/packages/compiler/src/ir/serialize.ts index 52c0994e3..cd3b161ec 100644 --- a/packages/compiler/src/ir/serialize.ts +++ b/packages/compiler/src/ir/serialize.ts @@ -5,7 +5,7 @@ */ import type { IrModule } from "./nodes.js"; -export const IR_VERSION = 4 as const; +export const IR_VERSION = 5 as const; export function serializeModule(mod: IrModule): string { return JSON.stringify(mod, (_key, value) => { diff --git a/packages/compiler/src/ir/validate.ts b/packages/compiler/src/ir/validate.ts index 875269a78..e1d57068a 100644 --- a/packages/compiler/src/ir/validate.ts +++ b/packages/compiler/src/ir/validate.ts @@ -1191,6 +1191,20 @@ export function validateModule(mod: IrModule): IrValidationError[] { errors.push({ message: `FFI binding "${entry.name}" has duplicate callback id "${param.callback.id}"`, loc: moduleLoc }); } ids.add(param.callback.id); + if (param.callback.invoke !== "script-thread" && param.callback.invoke !== "foreign") { + errors.push({ message: `FFI callback "${entry.name}:${param.callback.id}" has invalid invoke mode`, loc: moduleLoc }); + } + if (param.callback.invoke === "foreign") { + if (param.callback.lifetime !== "retained") { + errors.push({ message: `FFI foreign callback "${entry.name}:${param.callback.id}" is not retained`, loc: moduleLoc }); + } + if (param.callback.returns !== "void") { + errors.push({ message: `FFI foreign callback "${entry.name}:${param.callback.id}" does not return void`, loc: moduleLoc }); + } + if (!param.callback.params.some(isFfiContextParam)) { + errors.push({ message: `FFI foreign callback "${entry.name}:${param.callback.id}" has no context`, loc: moduleLoc }); + } + } if (param.callback.lifetime === "retained") { retainedFfiCallbacks.set(`${entry.name}:${param.callback.id}`, param.callback); } diff --git a/packages/compiler/src/library/int-infer.test.ts b/packages/compiler/src/library/int-infer.test.ts index 817aea6e9..236c2d94f 100644 --- a/packages/compiler/src/library/int-infer.test.ts +++ b/packages/compiler/src/library/int-infer.test.ts @@ -65,7 +65,7 @@ const sink = (name: string): IrFunction => ({ /** A module holding the case function plus the two declared sinks. */ function caseModule(params: string[], locals: string[], body: IrStmt[]): IrModule { return { - irVersion: 4, + irVersion: 5, sourceFile: "corpus.ts", functions: [ sink("send"), @@ -136,7 +136,7 @@ const RECORD_CFG: IntSlotConfig = { function recordCase(body: IrStmt[], names = ["m"], extraFns: IrFunction[] = []): IrModule { return { - irVersion: 4, + irVersion: 5, sourceFile: "fields.ts", functions: [ ...extraFns, @@ -177,7 +177,7 @@ const classCountRead = (): IrExpr => ({ function onlyOrdinaryClass(body: IrStmt[]): IntVerdict { const mod: IrModule = { - irVersion: 4, + irVersion: 5, sourceFile: "class-fields.ts", functions: [ sink("send"), @@ -391,7 +391,7 @@ describe("the domain's edges beyond the corpus", () => { loc, }; const mod: IrModule = { - irVersion: 4, + irVersion: 5, sourceFile: "optional.ts", functions: [{ name: "normalize", diff --git a/packages/compiler/test/bytes-element-emission.test.ts b/packages/compiler/test/bytes-element-emission.test.ts index e5be5ccd5..9b6b28980 100644 --- a/packages/compiler/test/bytes-element-emission.test.ts +++ b/packages/compiler/test/bytes-element-emission.test.ts @@ -66,7 +66,7 @@ function fixture(): IrModule { ); return { - irVersion: 4, + irVersion: 5, sourceFile: loc.file, entry: "__main", functions: [{ name: "__main", params: [], returnType: VOID, locals, body, loc }], @@ -136,7 +136,7 @@ function receiverReassignmentFixture(): IrModule { ]; return { - irVersion: 4, + irVersion: 5, sourceFile: loc.file, entry: "__main", functions: [{ name: "__main", params: [], returnType: VOID, locals, body, loc }], @@ -201,7 +201,7 @@ function integerLoopFixture(mutatesIndex = false): IrModule { { kind: "bytesSet", arr: bytesRef(), index: indexRef(), value: ref("sum"), loc }, ); return { - irVersion: 4, + irVersion: 5, sourceFile: loc.file, entry: "__main", functions: [{ diff --git a/packages/compiler/test/emit-c.test.ts b/packages/compiler/test/emit-c.test.ts index 435fa90dc..1bf9de600 100644 --- a/packages/compiler/test/emit-c.test.ts +++ b/packages/compiler/test/emit-c.test.ts @@ -35,7 +35,7 @@ test("strings: literals, concat in a loop, toString, RC-clean under audit", asyn // while (i < 3) { acc = acc + ("-" + i); i = i + 1; } // console.log(acc, acc === "x-0-1-2", "α∂" < "β"); const mod: IrModule = { - irVersion: 4, + irVersion: 5, sourceFile: "s.ts", entry: "__main", functions: [ @@ -113,7 +113,7 @@ test("short-circuit: right operand of && only evaluates when left is true", asyn // if (false && sideEffect()) {} ; if (true || sideEffect()) {} // console.log("done") const mod: IrModule = { - irVersion: 4, + irVersion: 5, sourceFile: "l.ts", entry: "__main", functions: [ @@ -161,7 +161,7 @@ test("string params: callee owns and releases; returns transfer ownership", asyn // function greet(who: string): string { return "hi " + who; } // console.log(greet("world")); const mod: IrModule = { - irVersion: 4, + irVersion: 5, sourceFile: "p.ts", entry: "__main", functions: [ diff --git a/packages/compiler/test/fixtures/fib-ir.ts b/packages/compiler/test/fixtures/fib-ir.ts index ec105ba21..3d5245608 100644 --- a/packages/compiler/test/fixtures/fib-ir.ts +++ b/packages/compiler/test/fixtures/fib-ir.ts @@ -18,7 +18,7 @@ const n = (localId: string): IrExpr => ({ kind: "varRef", localId, type: F64, lo const num = (value: number): IrExpr => ({ kind: "numLit", value, type: F64, loc }); export const fibModule: IrModule = { - irVersion: 4, + irVersion: 5, sourceFile: "fib.ts", entry: "__main", functions: [ diff --git a/packages/compiler/test/ir.test.ts b/packages/compiler/test/ir.test.ts index 14cb0c75a..ed119dbf7 100644 --- a/packages/compiler/test/ir.test.ts +++ b/packages/compiler/test/ir.test.ts @@ -22,7 +22,7 @@ test("fib module JSON round-trips", () => { test("validator rejects type mismatches and bad references", () => { const loc = { file: "t.ts", start: 0, end: 0 }; const bad: IrModule = { - irVersion: 4, + irVersion: 5, sourceFile: "t.ts", entry: "__main", functions: [ @@ -83,6 +83,6 @@ test("serializer round-trips ±Infinity and refuses NaN", () => { }); test("deserializer rejects the previous IR version", () => { - const json = serializeModule(fibModule).replace('"irVersion": 4', '"irVersion": 3'); + const json = serializeModule(fibModule).replace('"irVersion": 5', '"irVersion": 4'); expect(() => deserializeModule(json)).toThrow(/version mismatch/); }); diff --git a/packages/runtime/src/scr_async.c b/packages/runtime/src/scr_async.c index c67aef4b7..7f69595ac 100644 --- a/packages/runtime/src/scr_async.c +++ b/packages/runtime/src/scr_async.c @@ -2198,6 +2198,22 @@ void scr_loop_set_watch(bool (*pending)(void), void (*dispatch)(void), int (*pol scr_watch_pollfd_fn = pollfd; } +/* The foreign-FFI queue hook (scr_ffi.c when a format-5 descriptor exists). + * Its pending count includes live registrations (reffed by default) and + * queued deliveries; dispatch runs exactly one callback per loop turn. */ +static bool (*scr_ffi_pending_fn)(void) = NULL; +static bool (*scr_ffi_dispatch_fn)(void) = NULL; +static int (*scr_ffi_pollfd_fn)(void) = NULL; +static void (*scr_ffi_stop_fn)(void) = NULL; + +void scr_loop_set_ffi(bool (*pending)(void), bool (*dispatch)(void), + int (*pollfd)(void), void (*stop)(void)) { + scr_ffi_pending_fn = pending; + scr_ffi_dispatch_fn = dispatch; + scr_ffi_pollfd_fn = pollfd; + scr_ffi_stop_fn = stop; +} + /* The stream hook (scr_stream.c, when linked): `pending` keeps the loop * alive while deferred stream ticks exist, `dispatch` drains them at the * TOP of every turn — before the events/net stations, the closest @@ -2295,6 +2311,12 @@ bool scr_loop_run(ScrPromise *top_level) { if (scr_exc_pending()) return false; if (dispatched) continue; } + /* Foreign native callbacks are macrotasks. Deliver one, then restart at + * the microtask checkpoint before considering the next queued post. */ + if (scr_ffi_dispatch_fn != NULL && scr_ffi_dispatch_fn()) { + if (scr_exc_pending()) return false; + if (scr_ready_len > 0 || scr_nt_head != NULL || scr_nunhandled > 0) continue; + } /* Stream tick dispatch (scr_stream.c, when linked): the deferred * next-tick emissions ('data' flow kicks, 'readable'/'end'/'finish'/ * 'drain'/'error'/'close') fire now, FIRST — the nextTick station. @@ -2363,6 +2385,7 @@ bool scr_loop_run(ScrPromise *top_level) { (scr_net_pending_fn != NULL && scr_net_pending_fn()) || (scr_dgram_pending_fn != NULL && scr_dgram_pending_fn()) || (scr_watch_pending_fn != NULL && scr_watch_pending_fn()) || + (scr_ffi_pending_fn != NULL && scr_ffi_pending_fn()) || scr_fs_renames_pending(); if (held) { scr_children_poll(); @@ -2382,6 +2405,7 @@ bool scr_loop_run(ScrPromise *top_level) { bool net = scr_net_pending_fn != NULL && scr_net_pending_fn(); bool dgram = scr_dgram_pending_fn != NULL && scr_dgram_pending_fn(); bool watch = scr_watch_pending_fn != NULL && scr_watch_pending_fn(); + bool ffi = scr_ffi_pending_fn != NULL && scr_ffi_pending_fn(); bool renames = scr_fs_renames_pending(); /* Timer liveness counts only REF'd timers: an unref'd timer stays in * the heap (and fires if the loop runs on for other reasons) but does @@ -2389,7 +2413,7 @@ bool scr_loop_run(ScrPromise *top_level) { * Children follow the same rule: an unref'd child is still REAPED * while the loop runs (kids drives the sweeps and sleeps above) but * only reffed ones keep the process alive. */ - if (scr_reffed_timers == 0 && scr_reffed_immediates == 0 && !scr_children_reffed_pending() && !io && !events && !net && !dgram && !watch && !renames) break; + if (scr_reffed_timers == 0 && scr_reffed_immediates == 0 && !scr_children_reffed_pending() && !io && !events && !net && !dgram && !watch && !ffi && !renames) break; /* Sleep to the earliest deadline, then run every due timer (each may * enqueue microtasks, which the next iteration drains first). Who * sleeps depends on what is pending: @@ -2430,11 +2454,11 @@ bool scr_loop_run(ScrPromise *top_level) { * on EINTR), so they re-impose a coarser cap — bounded Ctrl-C and * socket latency during a fetch, without the reap-granularity * cost. */ - else if ((evw || net || dgram || watch) && due > now + SCR_SIGNAL_POLL_MS) due = now + SCR_SIGNAL_POLL_MS; + else if ((evw || net || dgram || watch || ffi) && due > now + SCR_SIGNAL_POLL_MS) due = now + SCR_SIGNAL_POLL_MS; scr_io_poll_fn(due > now ? due - now : 0); now = scr_now_ms(); if (scr_ready_len > 0) continue; /* io callbacks woke fibers */ - } else if (evw || net || dgram || watch) { + } else if (evw || net || dgram || watch || ffi) { #if defined(_WIN32) || defined(__wasi__) /* The win32 arm, and WASI hosts whose poll_oneoff adapters do not * reliably wake for a closed inherited stdin pipe: the sleep is a capped nanosleep and @@ -2449,6 +2473,7 @@ bool scr_loop_run(ScrPromise *top_level) { * WaitForMultipleObjects over WSAEVENTs, or IOCP. */ if (evw && due > now + SCR_SIGNAL_POLL_MS) due = now + SCR_SIGNAL_POLL_MS; if ((net || dgram || watch) && due > now + SCR_CHILD_POLL_MS) due = now + SCR_CHILD_POLL_MS; + if (ffi && due > now + SCR_CHILD_POLL_MS) due = now + SCR_CHILD_POLL_MS; if (kids && due > now + SCR_CHILD_POLL_MS) due = now + SCR_CHILD_POLL_MS; if (due > now) { double wait = due - now; @@ -2465,7 +2490,7 @@ bool scr_loop_run(ScrPromise *top_level) { * events are pending); unrepresentable children keep the ~1ms reap cap * instead. Dispatch happens at the next turn's top — the poll only * decides how long to sleep. */ - struct pollfd fds[6]; + struct pollfd fds[7]; int nfds = 0; int evfds[2]; int nev = evw && scr_events_pollfds_fn != NULL ? scr_events_pollfds_fn(evfds) : 0; @@ -2510,6 +2535,16 @@ bool scr_loop_run(ScrPromise *top_level) { due = now + SCR_SIGNAL_POLL_MS; } } + if (ffi) { + int ffd = scr_ffi_pollfd_fn != NULL ? scr_ffi_pollfd_fn() : -1; + if (ffd >= 0) { + fds[nfds].fd = ffd; + fds[nfds].events = POLLIN; + fds[nfds++].revents = 0; + } else if (due > now + SCR_CHILD_POLL_MS) { + due = now + SCR_CHILD_POLL_MS; + } + } if (kids) { int cfd = scr_children_wake_fd(); if (cfd >= 0) { @@ -2618,6 +2653,10 @@ bool scr_loop_run(ScrPromise *top_level) { * them as leaks. Uncaught callback throws still return above for main * to report through the existing exceptional teardown path. */ scr_timers_teardown(); + /* No reffed foreign registration or queued invocation remains at ordinary + * exhaustion. Disarm posting before exit listeners/global teardown so a + * straggling native thread becomes a safe silent drop. */ + if (scr_ffi_stop_fn != NULL) scr_ffi_stop_fn(); /* Retained native FFI registrations are deliberately NOT torn down * here: process 'exit' listeners run after the loop returns (inline in * main, before any atexit handler) and may legitimately release a diff --git a/packages/runtime/src/scr_ffi.c b/packages/runtime/src/scr_ffi.c index 6750cb78e..3be73ddf4 100644 --- a/packages/runtime/src/scr_ffi.c +++ b/packages/runtime/src/scr_ffi.c @@ -2,17 +2,32 @@ #include -/* Retained callbacks are same-thread only in format 4, so the registration - * ledger deliberately has no synchronization. The global list exists only - * to make every still-live registration release before the executable RC - * audit. A table remains linked after an explicit teardown so a later - * registration can reuse it without mutating the list twice. */ +/* The process-global retained-registration ledger. Format-4 script-thread + * tables use the lock-free path in this always-linked unit. Format-5 foreign + * tables install an optional teardown hook owned by scr_ffi_queue.c, keeping + * all queue/thread machinery out of unrelated binaries. */ static ScrFfiTable *scr_ffi_tables; static bool scr_ffi_exit_registered; static void scr_ffi_oom(void) { scr_trap("scriptc: out of memory\n"); } +void scr_ffi_link(ScrFfiTable *table) { + if (!table->linked) { + table->linked = true; + table->next = scr_ffi_tables; + scr_ffi_tables = table; + } + if (!scr_ffi_exit_registered) { + scr_ffi_exit_registered = true; + scr_atexit(scr_ffi_teardown_all); + } +} + void scr_ffi_teardown(ScrFfiTable *table) { + if (table->teardown != NULL) { + table->teardown(table); + return; + } /* Disarm the raw trampoline slot FIRST: a closure release below cannot * run script code today, but the slot must never dangle over freed * entries — a native exit-path invocation takes the NULL trap instead. */ @@ -33,15 +48,7 @@ void scr_ffi_teardown_all(void) { } void scr_ffi_retain(ScrFfiTable *table, ScrClosure *callback) { - if (!table->linked) { - table->linked = true; - table->next = scr_ffi_tables; - scr_ffi_tables = table; - } - if (!scr_ffi_exit_registered) { - scr_ffi_exit_registered = true; - scr_atexit(scr_ffi_teardown_all); - } + scr_ffi_link(table); if (table->len == table->cap) { size_t cap = table->cap == 0 ? 4 : table->cap * 2; if (cap < table->cap || cap > SIZE_MAX / sizeof *table->entries) scr_ffi_oom(); diff --git a/packages/runtime/src/scr_ffi_queue.c b/packages/runtime/src/scr_ffi_queue.c new file mode 100644 index 000000000..7ca9f2f4b --- /dev/null +++ b/packages/runtime/src/scr_ffi_queue.c @@ -0,0 +1,379 @@ +#include "scr_runtime.h" + +#include +#include +#include +#include + +#ifdef _WIN32 +#include +#elif defined(__wasi__) +/* Native FFI is rejected for WASI. This keeps direct runtime builds honest. */ +#else +#include +#include +#include +#endif + +typedef enum { + SCR_FFI_ARG_NONE = 0, + SCR_FFI_ARG_F64, + SCR_FFI_ARG_BOOL, + SCR_FFI_ARG_U8, + SCR_FFI_ARG_U32, + SCR_FFI_ARG_I32, + SCR_FFI_ARG_DATA, +} ScrFfiArgKind; + +typedef struct { + ScrFfiArgKind kind; + union { + double f64; + uint32_t u32; + int32_t i32; + struct { + uint8_t *data; + size_t len; + } span; + } value; +} ScrFfiArg; + +struct ScrFfiCall { + ScrFfiTable *table; /* opaque on the posting thread */ + void *loop; /* reserved instance/loop identity */ + ScrClosure *callback; /* opaque; the table owns its script RC pin */ + ScrFfiDispatchFn dispatch; + size_t nargs; + ScrFfiArg *args; + struct ScrFfiCall *next; +}; + +static ScrFfiCall *scr_ffi_head; +static ScrFfiCall *scr_ffi_tail; +static size_t scr_ffi_foreign_registrations; +static bool scr_ffi_installed; +static bool scr_ffi_stopping; +static int scr_ffi_wake_pipe[2] = {-1, -1}; + +#ifdef _WIN32 +static SRWLOCK scr_ffi_lock = SRWLOCK_INIT; +static void scr_ffi_lock_enter(void) { AcquireSRWLockExclusive(&scr_ffi_lock); } +static void scr_ffi_lock_leave(void) { ReleaseSRWLockExclusive(&scr_ffi_lock); } +#elif defined(__wasi__) +static void scr_ffi_lock_enter(void) {} +static void scr_ffi_lock_leave(void) {} +#else +static pthread_mutex_t scr_ffi_lock = PTHREAD_MUTEX_INITIALIZER; +static void scr_ffi_lock_enter(void) { (void)pthread_mutex_lock(&scr_ffi_lock); } +static void scr_ffi_lock_leave(void) { (void)pthread_mutex_unlock(&scr_ffi_lock); } +#endif + +static void scr_ffi_oom(void) { scr_trap("scriptc: out of memory\n"); } + +/* Foreign trampolines cannot enter the script exception machinery. Allocation + * failure and malformed native pointers are fatal process-boundary failures, + * reported without touching any runtime object. */ +static void scr_ffi_foreign_fatal(const char *message) { + fputs(message, stderr); + abort(); +} + +static void *scr_ffi_foreign_alloc(size_t size) { + void *ptr = malloc(size == 0 ? 1 : size); + if (ptr == NULL) scr_ffi_foreign_fatal("scriptc: out of memory\n"); + return ptr; +} + +static void scr_ffi_call_free(ScrFfiCall *call) { + if (call == NULL) return; + for (size_t i = 0; i < call->nargs; i++) { + if (call->args[i].kind == SCR_FFI_ARG_DATA) free(call->args[i].value.span.data); + } + free(call->args); + free(call); +} + +static bool scr_ffi_table_contains(const ScrFfiTable *table, const ScrClosure *callback) { + for (size_t i = 0; i < table->len; i++) { + if (table->entries[i] == callback) return true; + } + return false; +} + +static void scr_ffi_table_retire(ScrFfiTable *table, ScrClosure *callback) { + if (table->retired_len == table->retired_cap) { + size_t cap = table->retired_cap == 0 ? 4 : table->retired_cap * 2; + if (cap < table->retired_cap || cap > SIZE_MAX / sizeof *table->retired) scr_ffi_oom(); + ScrClosure **retired = realloc(table->retired, cap * sizeof *retired); + if (retired == NULL) scr_ffi_oom(); + table->retired = retired; + table->retired_cap = cap; + } + table->retired[table->retired_len++] = callback; +} + +static void scr_ffi_release_retired(ScrFfiTable *table) { + ScrClosure **retired = NULL; + size_t len = 0; + scr_ffi_lock_enter(); + if (table->queued == 0 && table->retired_len > 0) { + retired = table->retired; + len = table->retired_len; + table->retired = NULL; + table->retired_len = 0; + table->retired_cap = 0; + } + scr_ffi_lock_leave(); + for (size_t i = 0; i < len; i++) scr_closure_release(retired[i]); + free(retired); +} + +void scr_ffi_retain_foreign(ScrFfiTable *table, ScrClosure *callback) { + table->teardown = &scr_ffi_teardown_foreign; + scr_ffi_link(table); + scr_ffi_lock_enter(); + if (table->len == table->cap) { + size_t cap = table->cap == 0 ? 4 : table->cap * 2; + if (cap < table->cap || cap > SIZE_MAX / sizeof *table->entries) scr_ffi_oom(); + ScrClosure **entries = realloc(table->entries, cap * sizeof *entries); + if (entries == NULL) scr_ffi_oom(); + table->entries = entries; + table->cap = cap; + } + table->entries[table->len++] = scr_closure_retain(callback); + scr_ffi_foreign_registrations++; + scr_ffi_lock_leave(); +} + +void scr_ffi_require_foreign(ScrFfiTable *table, ScrClosure *callback) { + scr_ffi_lock_enter(); + bool found = scr_ffi_table_contains(table, callback); + scr_ffi_lock_leave(); + if (!found) scr_trap("scriptc: releasing a native callback registration that does not exist\n"); +} + +void scr_ffi_release_foreign(ScrFfiTable *table, ScrClosure *callback) { + scr_ffi_lock_enter(); + for (size_t i = 0; i < table->len; i++) { + if (table->entries[i] != callback) continue; + ScrClosure *owned = table->entries[i]; + table->len--; + if (i != table->len) table->entries[i] = table->entries[table->len]; + if (scr_ffi_foreign_registrations > 0) scr_ffi_foreign_registrations--; + bool deferred = table->queued > 0; + if (deferred) scr_ffi_table_retire(table, owned); + scr_ffi_lock_leave(); + if (!deferred) scr_closure_release(owned); + return; + } + scr_ffi_lock_leave(); + scr_trap("scriptc: releasing a native callback registration that does not exist\n"); +} + +ScrFfiCall *scr_ffi_call_new(ScrFfiTable *table, ScrClosure *callback, + ScrFfiDispatchFn dispatch, size_t nargs) { + ScrFfiCall *call = scr_ffi_foreign_alloc(sizeof *call); + call->table = table; + call->loop = table->loop; + call->callback = callback; + call->dispatch = dispatch; + call->nargs = nargs; + call->args = nargs == 0 ? NULL : scr_ffi_foreign_alloc(nargs * sizeof *call->args); + if (nargs > 0) memset(call->args, 0, nargs * sizeof *call->args); + call->next = NULL; + return call; +} + +static ScrFfiArg *scr_ffi_call_arg(ScrFfiCall *call, size_t index) { + if (index >= call->nargs) scr_ffi_foreign_fatal("scriptc: invalid staged FFI callback argument\n"); + return &call->args[index]; +} + +void scr_ffi_call_set_f64(ScrFfiCall *call, size_t i, double v) { + ScrFfiArg *arg = scr_ffi_call_arg(call, i); arg->kind = SCR_FFI_ARG_F64; arg->value.f64 = v; +} +void scr_ffi_call_set_bool(ScrFfiCall *call, size_t i, uint8_t v) { + ScrFfiArg *arg = scr_ffi_call_arg(call, i); arg->kind = SCR_FFI_ARG_BOOL; arg->value.u32 = v != 0; +} +void scr_ffi_call_set_u8(ScrFfiCall *call, size_t i, uint8_t v) { + ScrFfiArg *arg = scr_ffi_call_arg(call, i); arg->kind = SCR_FFI_ARG_U8; arg->value.u32 = v; +} +void scr_ffi_call_set_u32(ScrFfiCall *call, size_t i, uint32_t v) { + ScrFfiArg *arg = scr_ffi_call_arg(call, i); arg->kind = SCR_FFI_ARG_U32; arg->value.u32 = v; +} +void scr_ffi_call_set_i32(ScrFfiCall *call, size_t i, int32_t v) { + ScrFfiArg *arg = scr_ffi_call_arg(call, i); arg->kind = SCR_FFI_ARG_I32; arg->value.i32 = v; +} + +static void scr_ffi_call_copy_data(ScrFfiCall *call, size_t i, + const uint8_t *v, size_t len, + const char *null_message) { + if (v == NULL && len != 0) scr_ffi_foreign_fatal(null_message); + ScrFfiArg *arg = scr_ffi_call_arg(call, i); + arg->kind = SCR_FFI_ARG_DATA; + arg->value.span.len = len; + arg->value.span.data = len == 0 ? NULL : scr_ffi_foreign_alloc(len); + if (len > 0) memcpy(arg->value.span.data, v, len); +} + +void scr_ffi_call_copy_string(ScrFfiCall *call, size_t i, + const uint8_t *v, size_t len) { + scr_ffi_call_copy_data(call, i, v, len, + "scriptc: native callback passed a NULL string span with nonzero length\n"); +} + +void scr_ffi_call_copy_bytes(ScrFfiCall *call, size_t i, + const uint8_t *v, size_t len) { + scr_ffi_call_copy_data(call, i, v, len, + "scriptc: native callback passed a NULL bytes span with nonzero length\n"); +} + +void scr_ffi_call_copy_cstring(ScrFfiCall *call, size_t i, const char *v) { + if (v == NULL) scr_ffi_foreign_fatal("scriptc: native callback passed a NULL cstring\n"); + scr_ffi_call_copy_data(call, i, (const uint8_t *)v, strlen(v), + "scriptc: native callback passed a NULL cstring\n"); +} + +void scr_ffi_post(ScrFfiCall *call) { + scr_ffi_lock_enter(); + if (scr_ffi_stopping || !scr_ffi_table_contains(call->table, call->callback)) { + scr_ffi_lock_leave(); + scr_ffi_call_free(call); + return; + } + call->table->queued++; + if (scr_ffi_tail != NULL) scr_ffi_tail->next = call; + else scr_ffi_head = call; + scr_ffi_tail = call; +#if !defined(_WIN32) && !defined(__wasi__) + if (scr_ffi_wake_pipe[1] >= 0) { + ssize_t ignored = write(scr_ffi_wake_pipe[1], "f", 1); + (void)ignored; /* a full pipe is already readable */ + } +#endif + scr_ffi_lock_leave(); +} + +static const ScrFfiArg *scr_ffi_get_arg(const ScrFfiCall *call, size_t index) { + if (index >= call->nargs) scr_trap("scriptc: invalid staged FFI callback argument\n"); + return &call->args[index]; +} + +double scr_ffi_call_get_f64(const ScrFfiCall *call, size_t i) { return scr_ffi_get_arg(call, i)->value.f64; } +bool scr_ffi_call_get_bool(const ScrFfiCall *call, size_t i) { return scr_ffi_get_arg(call, i)->value.u32 != 0; } +double scr_ffi_call_get_u8(const ScrFfiCall *call, size_t i) { return (double)scr_ffi_get_arg(call, i)->value.u32; } +double scr_ffi_call_get_u32(const ScrFfiCall *call, size_t i) { return (double)scr_ffi_get_arg(call, i)->value.u32; } +double scr_ffi_call_get_i32(const ScrFfiCall *call, size_t i) { return (double)scr_ffi_get_arg(call, i)->value.i32; } +const uint8_t *scr_ffi_call_get_data(const ScrFfiCall *call, size_t i) { return scr_ffi_get_arg(call, i)->value.span.data; } +size_t scr_ffi_call_get_len(const ScrFfiCall *call, size_t i) { return scr_ffi_get_arg(call, i)->value.span.len; } + +static bool scr_ffi_pending(void) { + bool pending; + scr_ffi_lock_enter(); + pending = scr_ffi_foreign_registrations > 0 || scr_ffi_head != NULL; + scr_ffi_lock_leave(); + return pending; +} + +static void scr_ffi_wake_drain(void) { +#if !defined(_WIN32) && !defined(__wasi__) + if (scr_ffi_wake_pipe[0] < 0) return; + char buf[64]; + while (read(scr_ffi_wake_pipe[0], buf, sizeof buf) > 0) {} +#endif +} + +static bool scr_ffi_dispatch(void) { + scr_ffi_lock_enter(); + ScrFfiCall *call = scr_ffi_head; + if (call != NULL) { + scr_ffi_head = call->next; + if (scr_ffi_head == NULL) scr_ffi_tail = NULL; + } + scr_ffi_lock_leave(); + if (call == NULL) return false; + + call->dispatch(call->callback, call); + + ScrFfiTable *table = call->table; + scr_ffi_lock_enter(); + if (table->queued > 0) table->queued--; + /* Keep the wake fd readable while queued calls remain. Drain only while + * holding the same lock writers use, closing the post-vs-drain lost-wakeup + * race for the transition to an empty queue. */ + bool more = scr_ffi_head != NULL; + if (!more) scr_ffi_wake_drain(); + scr_ffi_lock_leave(); + scr_ffi_call_free(call); + scr_ffi_release_retired(table); + return true; +} + +static int scr_ffi_pollfd(void) { return scr_ffi_wake_pipe[0]; } + +void scr_ffi_install(void) { + if (scr_ffi_installed) return; + scr_ffi_installed = true; +#if !defined(_WIN32) && !defined(__wasi__) + if (pipe(scr_ffi_wake_pipe) == 0) { + for (int i = 0; i < 2; i++) { + (void)fcntl(scr_ffi_wake_pipe[i], F_SETFL, O_NONBLOCK); + (void)fcntl(scr_ffi_wake_pipe[i], F_SETFD, FD_CLOEXEC); + } + } else { + scr_ffi_wake_pipe[0] = scr_ffi_wake_pipe[1] = -1; + } +#endif + scr_loop_set_ffi(&scr_ffi_pending, &scr_ffi_dispatch, &scr_ffi_pollfd, + &scr_ffi_stop); +} + +void scr_ffi_stop(void) { + scr_ffi_lock_enter(); + if (scr_ffi_stopping) { + scr_ffi_lock_leave(); + return; + } + scr_ffi_stopping = true; + ScrFfiCall *calls = scr_ffi_head; + scr_ffi_head = scr_ffi_tail = NULL; + for (ScrFfiCall *call = calls; call != NULL; call = call->next) { + if (call->table->queued > 0) call->table->queued--; + } + scr_ffi_wake_drain(); +#if !defined(_WIN32) && !defined(__wasi__) + if (scr_ffi_wake_pipe[0] >= 0) close(scr_ffi_wake_pipe[0]); + if (scr_ffi_wake_pipe[1] >= 0) close(scr_ffi_wake_pipe[1]); +#endif + scr_ffi_wake_pipe[0] = scr_ffi_wake_pipe[1] = -1; + scr_ffi_lock_leave(); + while (calls != NULL) { + ScrFfiCall *next = calls->next; + ScrFfiTable *table = calls->table; + scr_ffi_call_free(calls); + scr_ffi_release_retired(table); + calls = next; + } +} + +void scr_ffi_teardown_foreign(ScrFfiTable *table) { + scr_ffi_stop(); + scr_ffi_lock_enter(); + ScrClosure **entries = table->entries; + size_t len = table->len; + ScrClosure **retired = table->retired; + size_t retired_len = table->retired_len; + if (scr_ffi_foreign_registrations >= len) scr_ffi_foreign_registrations -= len; + table->entries = NULL; + table->len = 0; + table->cap = 0; + table->retired = NULL; + table->retired_len = 0; + table->retired_cap = 0; + table->queued = 0; + scr_ffi_lock_leave(); + for (size_t i = 0; i < len; i++) scr_closure_release(entries[i]); + free(entries); + for (size_t i = 0; i < retired_len; i++) scr_closure_release(retired[i]); + free(retired); +} diff --git a/packages/runtime/src/scr_runtime.h b/packages/runtime/src/scr_runtime.h index 08b6e0b55..d87732777 100644 --- a/packages/runtime/src/scr_runtime.h +++ b/packages/runtime/src/scr_runtime.h @@ -1327,8 +1327,10 @@ void scr_closure_release(ScrClosure *c); /* releases the boxes; NULL-tolerant */ * every pin the new registration superseded — a duplicate of the same * closure included — so after any number of set calls exactly one * release is pending. The table owns one closure reference per entry and - * joins a process-global teardown list on first use. Retained callbacks - * do not contribute event-loop liveness. */ + * joins a process-global teardown list on first use. Script-thread retained + * callbacks do not contribute event-loop liveness; foreign registrations do, + * and their queued invocations keep released closure pins alive until the + * script-thread dispatch has finished. */ typedef struct ScrFfiTable { ScrClosure **entries; size_t len; @@ -1340,9 +1342,26 @@ typedef struct ScrFfiTable { * late native invocation hits the trampoline's NULL trap instead of a * freed closure. NULL for context-bearing descriptors. */ ScrClosure **slot; + /* Foreign descriptors only. The global FFI queue lock protects these + * fields together with entries/len/cap. Retired pins were explicitly + * released while a queued invocation could still name them; dispatch + * drops them on the script thread after this table's queue reaches zero. */ + ScrClosure **retired; + size_t retired_len; + size_t retired_cap; + size_t queued; + /* Reserved process-loop identity captured by queued calls. Executables use + * one global loop today; library mode can make this per instance later + * without changing the post/call ABI. */ + void *loop; + /* Optional format-5 teardown. NULL keeps format-4 tables on the compact + * always-linked path; foreign tables point into scr_ffi_queue.c. */ + void (*teardown)(struct ScrFfiTable *table); } ScrFfiTable; +void scr_ffi_link(ScrFfiTable *table); void scr_ffi_retain(ScrFfiTable *table, ScrClosure *callback); +void scr_ffi_retain_foreign(ScrFfiTable *table, ScrClosure *callback); /* Raw singleton registration, split around the native set call: * retain_slot pins the incoming closure BEFORE the call without touching * the current registration; commit_slot repoints the slot and retires the @@ -1353,9 +1372,47 @@ void scr_ffi_commit_slot(ScrFfiTable *table, ScrClosure *callback); * call so a bogus release cannot reach native code. */ void scr_ffi_require(ScrFfiTable *table, ScrClosure *callback); void scr_ffi_release(ScrFfiTable *table, ScrClosure *callback); +void scr_ffi_require_foreign(ScrFfiTable *table, ScrClosure *callback); +void scr_ffi_release_foreign(ScrFfiTable *table, ScrClosure *callback); void scr_ffi_teardown(ScrFfiTable *table); void scr_ffi_teardown_all(void); +/* Format-5 foreign-thread delivery. A generated native trampoline creates a + * plain malloc-backed call, stages scalar values or copied byte spans, and + * posts it. It never touches closure RC, exception cells, fibers, or other + * script-thread-only runtime state. The generated dispatch thunk runs later + * on the event loop and materializes script strings/bytes there. */ +typedef struct ScrFfiCall ScrFfiCall; +typedef void (*ScrFfiDispatchFn)(ScrClosure *callback, ScrFfiCall *call); + +ScrFfiCall *scr_ffi_call_new(ScrFfiTable *table, ScrClosure *callback, + ScrFfiDispatchFn dispatch, size_t nargs); +void scr_ffi_call_set_f64(ScrFfiCall *call, size_t index, double value); +void scr_ffi_call_set_bool(ScrFfiCall *call, size_t index, uint8_t value); +void scr_ffi_call_set_u8(ScrFfiCall *call, size_t index, uint8_t value); +void scr_ffi_call_set_u32(ScrFfiCall *call, size_t index, uint32_t value); +void scr_ffi_call_set_i32(ScrFfiCall *call, size_t index, int32_t value); +void scr_ffi_call_copy_cstring(ScrFfiCall *call, size_t index, const char *value); +void scr_ffi_call_copy_string(ScrFfiCall *call, size_t index, + const uint8_t *value, size_t len); +void scr_ffi_call_copy_bytes(ScrFfiCall *call, size_t index, + const uint8_t *value, size_t len); +void scr_ffi_post(ScrFfiCall *call); + +double scr_ffi_call_get_f64(const ScrFfiCall *call, size_t index); +bool scr_ffi_call_get_bool(const ScrFfiCall *call, size_t index); +double scr_ffi_call_get_u8(const ScrFfiCall *call, size_t index); +double scr_ffi_call_get_u32(const ScrFfiCall *call, size_t index); +double scr_ffi_call_get_i32(const ScrFfiCall *call, size_t index); +const uint8_t *scr_ffi_call_get_data(const ScrFfiCall *call, size_t index); +size_t scr_ffi_call_get_len(const ScrFfiCall *call, size_t index); + +void scr_ffi_install(void); +void scr_ffi_stop(void); +void scr_ffi_teardown_foreign(ScrFfiTable *table); +void scr_loop_set_ffi(bool (*pending)(void), bool (*dispatch)(void), + int (*pollfd)(void), void (*stop)(void)); + /* ── unions ───────────────────────────────────────────────────────── * A union value (`A | B`) is an IMMUTABLE tagged box: a refcounted header, * the arm's tag (its index in the compiler's canonical arm order), and one diff --git a/tests/ffi/main.ts b/tests/ffi/main.ts index c6a9c0ec5..3f39380a0 100644 --- a/tests/ffi/main.ts +++ b/tests/ffi/main.ts @@ -210,6 +210,58 @@ nativeRetainedAdd((_value: number) => { if (exitCapture.length === 0) console.log("unreachable"); }); +declare function nativeForeignStart( + callback: (value: number, label: string) => void, +): void; +declare function nativeForeignStop( + callback: (value: number, label: string) => void, +): void; +declare function nativeForeignBurstStart( + callback: (threadId: number, sequence: number) => void, +): void; +declare function nativeForeignBurstStop( + callback: (threadId: number, sequence: number) => void, +): void; + +const foreignEvents: string[] = []; +let foreignBurstDone = false; +const foreignTick = (value: number, label: string) => { + foreignEvents.push(`${value}:${label}`); + if (value === 3) { + nativeForeignStop(foreignTick); + if (foreignBurstDone) console.log(foreignEvents.join("|")); + } +}; +nativeForeignStart(foreignTick); + +let foreignBurstCount = 0; +let foreignBurstSum = 0; +let foreignTimerTicks = 0; +let foreignBurstOrderErrors = 0; +const foreignBurstNext = [0, 0]; +const foreignTimer = setInterval(() => { + foreignTimerTicks++; +}, 0); +const foreignBurstTick = (threadId: number, sequence: number) => { + if (sequence !== foreignBurstNext[threadId]) foreignBurstOrderErrors++; + foreignBurstNext[threadId] = foreignBurstNext[threadId] + 1; + foreignBurstCount++; + foreignBurstSum += threadId * 500 + sequence; + if (foreignBurstCount === 1000) { + nativeForeignBurstStop(foreignBurstTick); + clearInterval(foreignTimer); + console.log( + foreignBurstCount, + foreignBurstSum, + foreignTimerTicks > 0, + foreignBurstOrderErrors, + ); + foreignBurstDone = true; + if (foreignEvents.length === 3) console.log(foreignEvents.join("|")); + } +}; +nativeForeignBurstStart(foreignBurstTick); + try { nativeCallbackStringThrow((value) => { throw new Error(`string callback boom: ${value}`); diff --git a/tests/ffi/native.c b/tests/ffi/native.c index 741e1589e..aefaee13f 100644 --- a/tests/ffi/native.c +++ b/tests/ffi/native.c @@ -1,6 +1,18 @@ +#ifndef _WIN32 +#define _POSIX_C_SOURCE 200809L +#endif + #include #include +#ifdef _WIN32 +#include +#include +#else +#include +#include +#endif + static double last_note; double sf_scale(double value) { @@ -201,3 +213,130 @@ void sf_retained_raw_remove(sf_retained_raw_cb callback) { void sf_retained_raw_pump(double value) { if (retained_raw != NULL) retained_raw(value); } + +/* Format-5 foreign-thread callbacks. The first fixture pins wake/FIFO/cstring + * copying; the second posts concurrently from two library-owned threads and + * is large enough for the script timer fairness assertion. */ +typedef void (*sf_foreign_cb)(double value, const char *label, void *context); +typedef void (*sf_foreign_burst_cb)(double thread_id, double sequence, + void *context); + +typedef struct { + sf_foreign_cb callback; + void *context; +#ifdef _WIN32 + HANDLE thread; +#else + pthread_t thread; +#endif +} sf_foreign_state; + +static sf_foreign_state foreign_state; + +#ifdef _WIN32 +static unsigned __stdcall sf_foreign_worker(void *opaque) { +#else +static void *sf_foreign_worker(void *opaque) { +#endif + sf_foreign_state *state = opaque; +#ifdef _WIN32 + Sleep(20); +#else + struct timespec delay = {0, 20 * 1000 * 1000}; + (void)nanosleep(&delay, NULL); +#endif + for (int i = 1; i <= 3; i++) { + char label[] = "foreign-copy"; + state->callback((double)i, label, state->context); + label[0] = 'x'; /* queued text must already own its copy */ + } +#ifdef _WIN32 + return 0; +#else + return NULL; +#endif +} + +void sf_foreign_start(sf_foreign_cb callback, void *context) { + foreign_state.callback = callback; + foreign_state.context = context; +#ifdef _WIN32 + foreign_state.thread = (HANDLE)_beginthreadex(NULL, 0, sf_foreign_worker, + &foreign_state, 0, NULL); +#else + (void)pthread_create(&foreign_state.thread, NULL, sf_foreign_worker, + &foreign_state); +#endif +} + +void sf_foreign_stop(sf_foreign_cb callback, void *context) { + (void)callback; + (void)context; +#ifdef _WIN32 + WaitForSingleObject(foreign_state.thread, INFINITE); + CloseHandle(foreign_state.thread); +#else + (void)pthread_join(foreign_state.thread, NULL); +#endif + foreign_state.callback = NULL; + foreign_state.context = NULL; +} + +typedef struct { + sf_foreign_burst_cb callback; + void *context; + int id; +#ifdef _WIN32 + HANDLE thread; +#else + pthread_t thread; +#endif +} sf_foreign_burst_state; + +static sf_foreign_burst_state foreign_burst[2]; + +#ifdef _WIN32 +static unsigned __stdcall sf_foreign_burst_worker(void *opaque) { +#else +static void *sf_foreign_burst_worker(void *opaque) { +#endif + sf_foreign_burst_state *state = opaque; + for (int i = 0; i < 500; i++) { + state->callback((double)state->id, (double)i, state->context); + } +#ifdef _WIN32 + return 0; +#else + return NULL; +#endif +} + +void sf_foreign_burst_start(sf_foreign_burst_cb callback, void *context) { + for (int i = 0; i < 2; i++) { + foreign_burst[i].callback = callback; + foreign_burst[i].context = context; + foreign_burst[i].id = i; +#ifdef _WIN32 + foreign_burst[i].thread = (HANDLE)_beginthreadex( + NULL, 0, sf_foreign_burst_worker, &foreign_burst[i], 0, NULL); +#else + (void)pthread_create(&foreign_burst[i].thread, NULL, + sf_foreign_burst_worker, &foreign_burst[i]); +#endif + } +} + +void sf_foreign_burst_stop(sf_foreign_burst_cb callback, void *context) { + (void)callback; + (void)context; + for (int i = 0; i < 2; i++) { +#ifdef _WIN32 + WaitForSingleObject(foreign_burst[i].thread, INFINITE); + CloseHandle(foreign_burst[i].thread); +#else + (void)pthread_join(foreign_burst[i].thread, NULL); +#endif + foreign_burst[i].callback = NULL; + foreign_burst[i].context = NULL; + } +} diff --git a/tests/ffi/profile.json b/tests/ffi/profile.json index 59993c502..e10ce9aa1 100644 --- a/tests/ffi/profile.json +++ b/tests/ffi/profile.json @@ -1,5 +1,5 @@ { - "ffi_format": 4, + "ffi_format": 5, "functions": [ { "name": "nativeScale", "symbol": "sf_scale", "params": ["f64"], "returns": "f64" }, { "name": "nativeInvert", "symbol": "sf_invert", "params": ["bool"], "returns": "bool" }, @@ -248,6 +248,58 @@ "symbol": "sf_retained_raw_pump", "params": ["f64"], "returns": "void" + }, + { + "name": "nativeForeignStart", + "symbol": "sf_foreign_start", + "params": [ + { + "callback": { + "id": "tick", + "params": ["f64", "cstring", { "context": "tick" }], + "returns": "void", + "lifetime": "retained", + "invoke": "foreign" + } + }, + { "context": "tick" } + ], + "returns": "void" + }, + { + "name": "nativeForeignStop", + "symbol": "sf_foreign_stop", + "params": [ + { "callback": { "release": "nativeForeignStart:tick" } }, + { "context": "nativeForeignStart:tick" } + ], + "returns": "void" + }, + { + "name": "nativeForeignBurstStart", + "symbol": "sf_foreign_burst_start", + "params": [ + { + "callback": { + "id": "tick", + "params": ["f64", "f64", { "context": "tick" }], + "returns": "void", + "lifetime": "retained", + "invoke": "foreign" + } + }, + { "context": "tick" } + ], + "returns": "void" + }, + { + "name": "nativeForeignBurstStop", + "symbol": "sf_foreign_burst_stop", + "params": [ + { "callback": { "release": "nativeForeignBurstStart:tick" } }, + { "context": "nativeForeignBurstStart:tick" } + ], + "returns": "void" } ], "libraries": [], diff --git a/tests/harness/ffi.test.ts b/tests/harness/ffi.test.ts index 58f6ba5ec..feada28a4 100644 --- a/tests/harness/ffi.test.ts +++ b/tests/harness/ffi.test.ts @@ -4,8 +4,9 @@ * result bytes must match. The fixture covers every value ABI class, * integer coercion, embedded-NUL/UTF-8 string spans, byte spans, format-2 * raw/context callbacks, format-3 callback cstring/span copies (lossy UTF-8, - * exact bytes, empty spans, ownership, and catchable throws), and format-4 - * retained registration/release ownership. */ + * exact bytes, empty spans, ownership, and catchable throws), format-4 + * retained registration/release ownership, and format-5 foreign-thread + * queue wakeup, concurrency, FIFO, fairness, string staging, and liveness. */ import { execFileSync, spawnSync } from "node:child_process"; import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; @@ -78,11 +79,13 @@ const expected = [ "lead:6|lead:7", "first:11|first:-1|second:12", "caught string callback boom: materialized", + "1000 499500 true 0", + "1:foreign-copy|2:foreign-copy|3:foreign-copy", "", ].join("\n"); describe.each(["c", "llvm"] as const)("outbound native FFI, %s backend", (backend) => { - test("calls the manifest-bound archive across every v1 ABI class plus v2/v3/v4 callback ABI classes", async () => { + test("calls the manifest-bound archive across every v1 ABI class plus v2/v3/v4/v5 callback ABI classes", async () => { const outDir = join(cacheRoot, backend); mkdirSync(outDir, { recursive: true }); const result = await compile(join(fixtureRoot, "main.ts"), { @@ -149,6 +152,131 @@ describe.each(["c", "llvm"] as const)("outbound native FFI, %s backend", (backen expect(run.status).not.toBe(0); expect(run.stderr).toContain("scriptc: native callback passed a NULL cstring"); }); + + test("a throw from marshalled delivery follows timer-style uncaught behavior and drains safely", async () => { + const outDir = join(cacheRoot, `foreign-throw-${backend}`); + mkdirSync(outDir, { recursive: true }); + const entry = join(outDir, "main.ts"); + const profilePath = join(outDir, "profile.json"); + writeFileSync( + entry, + [ + "declare function nativeForeignStart(callback: (value: number, label: string) => void): void;", + "nativeForeignStart((value, _label) => {", + " console.log('foreign-before-throw', value);", + " throw new Error('foreign boom');", + "});", + "", + ].join("\n"), + ); + writeFileSync( + profilePath, + JSON.stringify({ + ffi_format: 5, + functions: [{ + name: "nativeForeignStart", + symbol: "sf_foreign_start", + params: [{ + callback: { + id: "tick", + params: ["f64", "cstring", { context: "tick" }], + returns: "void", + lifetime: "retained", + invoke: "foreign", + }, + }, { context: "tick" }], + returns: "void", + }], + libraries: [nativeArchive()], + }), + ); + const result = await compile(entry, { + outDir, + outPath: join(outDir, "program"), + backend, + sanitize, + ffiProfilePath: profilePath, + }); + if (!result.ok) { + throw new Error(result.diagnostics.map((d) => `${d.code}: ${d.message}`).join("\n")); + } + const run = spawnSync(result.binaryPath, [], { encoding: "utf8" }); + expect(run.status).toBe(1); + expect(run.stdout).toBe("foreign-before-throw 1\n"); + expect(run.stderr).toContain("Uncaught Error: foreign boom"); + }); + + test.runIf(process.platform !== "win32")( + "drains next-ticks before another ready event-loop station", + async () => { + const outDir = join(cacheRoot, `foreign-nexttick-${backend}`); + mkdirSync(outDir, { recursive: true }); + const entry = join(outDir, "main.ts"); + const profilePath = join(outDir, "profile.json"); + writeFileSync( + entry, + [ + "declare function nativeForeignStart(callback: (value: number, label: string) => void): void;", + "declare function nativeForeignStop(callback: (value: number, label: string) => void): void;", + "process.once('SIGTERM', () => console.log('signal'));", + "const tick = (value: number, _label: string) => {", + " if (value === 1) {", + " console.log('callback');", + " process.kill(process.pid, 'SIGTERM');", + " process.nextTick(() => console.log('nextTick'));", + // Keep the loop alive long enough for Node-style signal delivery. + " setTimeout(() => {}, 50);", + " }", + " if (value === 3) nativeForeignStop(tick);", + "};", + "nativeForeignStart(tick);", + "", + ].join("\n"), + ); + writeFileSync( + profilePath, + JSON.stringify({ + ffi_format: 5, + functions: [{ + name: "nativeForeignStart", + symbol: "sf_foreign_start", + params: [{ + callback: { + id: "tick", + params: ["f64", "cstring", { context: "tick" }], + returns: "void", + lifetime: "retained", + invoke: "foreign", + }, + }, { context: "tick" }], + returns: "void", + }, { + name: "nativeForeignStop", + symbol: "sf_foreign_stop", + params: [{ callback: { release: "nativeForeignStart:tick" } }, { + context: "nativeForeignStart:tick", + }], + returns: "void", + }], + libraries: [nativeArchive()], + }), + ); + const result = await compile(entry, { + outDir, + outPath: join(outDir, "program"), + backend, + sanitize, + ffiProfilePath: profilePath, + }); + if (!result.ok) { + throw new Error(result.diagnostics.map((d) => `${d.code}: ${d.message}`).join("\n")); + } + const run = spawnSync(result.binaryPath, [], { encoding: "utf8" }); + expect(run.stderr).toBe(""); + expect(run.status).toBe(0); + expect(run.stdout).toBe("callback\nnextTick\nsignal\n"); + }, + ); }); test("manifest validation is strict and source-facing", () => { @@ -345,6 +473,90 @@ test.each([ }, message: "registered by the same call", }, + { + name: "a foreign callback before format 5", + profile: { + ffi_format: 4, + functions: [{ + name: "visit", + symbol: "sf_visit", + params: [{ + callback: { + id: "visit", + params: [{ context: "visit" }], + returns: "void", + lifetime: "retained", + invoke: "foreign", + }, + }, { context: "visit" }], + returns: "void", + }], + }, + message: "value 'foreign' requires ffi_format 5", + }, + { + name: "a call-scoped foreign callback", + profile: { + ffi_format: 5, + functions: [{ + name: "visit", + symbol: "sf_visit", + params: [{ + callback: { + id: "visit", + params: [{ context: "visit" }], + returns: "void", + lifetime: "call", + invoke: "foreign", + }, + }, { context: "visit" }], + returns: "void", + }], + }, + message: "requires lifetime 'retained'", + }, + { + name: "a value-returning foreign callback", + profile: { + ffi_format: 5, + functions: [{ + name: "visit", + symbol: "sf_visit", + params: [{ + callback: { + id: "visit", + params: [{ context: "visit" }], + returns: "f64", + lifetime: "retained", + invoke: "foreign", + }, + }, { context: "visit" }], + returns: "void", + }], + }, + message: "requires returns 'void'", + }, + { + name: "a context-free foreign callback", + profile: { + ffi_format: 5, + functions: [{ + name: "visit", + symbol: "sf_visit", + params: [{ + callback: { + id: "visit", + params: ["f64"], + returns: "void", + lifetime: "retained", + invoke: "foreign", + }, + }], + returns: "void", + }], + }, + message: "requires a context entry", + }, { name: "a release carrying its own signature", profile: {