Skip to content

Latest commit

 

History

History
156 lines (77 loc) · 30.8 KB

dotnet-backends.md

File metadata and controls

156 lines (77 loc) · 30.8 KB

C# and .NET backends

FScript.Backend.CSharp emits a C# project from validated LIR. FScript.Backend.DotNet has a separate assembly optimization pipeline and uses Roslyn for verified PE emission. Generated code contains ordinary compiled methods and typed delegates where functions escape, and depends on BCL support helpers rather than the AST evaluator, LIR interpreter, or ScriptHost.

Both backends share LirMonomorphization and LirSimplification. Reachable polymorphic uses receive concrete instantiations; reachable open results and effectful generalized initializers receive diagnostics.

For target selection and executable workflows, see Compilation. The user-facing optimization guide compares both targets and illustrates the limits described below.

Purity contracts

PurityContracts contains PureFunctions and PureTypes sets of registered script names. The compiler API's explicit contract overloads default to empty sets through legacy entry points. The CLI discovers optional FScriptPureFunctions/FScriptPureTypes provider methods returning IEnumerable<string>. Function contracts include totality and effects; type contracts include deep immutability and safe registered-field reads. Provider initialization and callbacks must satisfy the documented conditional guarantee. Declarations are trusted; protected runtime contracts cannot be overridden.

LirEffects performs conservative contract-aware analysis shared by specialization and simplification. Recursive summaries start observable, avoiding circular proofs of termination. Unproven failures, non-exhaustive matches, external reads without type guarantees, and unknown callback effects remain observable. Pure calls can be pruned only with pure arguments and a correctly retained application boundary. Generated-member/dependency pruning then removes newly unreachable code. This does not change the AST evaluator or SDK validation.

Assembly optimization

Both native consumers run LirCommonSubexpressions.optimize after specialization, simplification, and any target-specific LIR reduction. Dominance-based value numbering reuses identical typed pure expressions through lexical bindings and initialized globals. Keys omit source spans and retain exact floating-point bits. Trusted external-type immutability is required for external value observations and dependencies. Function- or task-containing result values are not shared, including aggregates whose elements contain these handles. Observable operations clear availability; branches, short-circuit operands, match guards, and loop bodies maintain separate conditional scopes. Earlier effectful guards invalidate availability for later alternatives. Function bodies start with empty availability and do not acquire caches across invocations. Administrative temporaries that have no sharing purpose are removed. The output uses existing LIR nodes and is validated before emission; artifacts and interpreted entry points do not enable this pass automatically. No memoization or runtime cache is introduced.

The .NET assembly pipeline additionally beta-reduces known nonrecursive calls with fresh lexical identities, propagates known partial applications, and hoists their top-level captures so effects execute once in source order. Recursive components are detected before expansion. Saturated calls use methods with multiple typed parameters. Exported or genuinely escaping functions retain typed delegates.

Saturated self calls in tail position become loops, including calls behind lets, branches, and pattern decisions. Next arguments are evaluated into temporaries before parameter reassignment. Escaping closures snapshot captures to preserve each iteration's immutable values. Mutual recursion and calls with pending work remain direct recursive calls.

Known list callbacks use typed loops and direct calls for mapping, iteration, folding, filtering, choosing, collecting, existence tests, and searches. Immediate helper blocks become static methods instead of delegate invocations. Constant boolean terms and branches are simplified while preserving observable evaluation. The CLR JIT can inline small native helpers.

Reachability starts at Run, result formatting, exported bindings, and the executable entry point. Unused generated methods, fields, and support types are removed. Reached constructors and interface implementations remain available. Native aggregates use immutable classes/structs without synthesized CLR record printing, equality, cloning, or deconstruction; FScript equality and formatting use dedicated reachable helpers. Ordinary assembly generation preserves provider assemblies; executable publishing can trim them as described below.

These passes apply only to DotNetBackend.generate. The C# target, serialized LIR, and LIR interpreter retain their existing optimization policy. Roslyn remains the final IL emitter; FScript makes the optimization decisions before emission.

Namespace qualification is not a reachability edge to a namespace's syntax body. Reaching a generated type or method through its fully qualified name retains that declaration and its actual dependencies, without retaining unrelated stdlib members in the same namespace. This rule applies to both libraries and executables.

Public API

  • CSharpBackend.generate : CSharpOptions -> Lir.Program -> Result<GeneratedFile list, CompilerDiagnostic list>
  • DotNetBackend.generate : DotNetOptions -> Lir.Program -> Result<GeneratedBinary list, CompilerDiagnostic list>
  • DotNetBackend.generateExecutable : DotNetExecutableOptions -> DotNetOptions -> Lir.Program -> Result<GeneratedBinary list, CompilerDiagnostic list>
  • ClrBindings.bindMethod : string -> MethodBase -> Lir.Program -> Lir.Program
  • ClrBindings.bindType : string -> System.Type -> Lir.Program -> Lir.Program

CLR linkage is serializable. Exact public overloads, instance receivers, and constructors are explicit. Native type projections use their actual CLR type. Existing extension providers use ClrBindings.bindProvider: generated typed adapters marshal values through FScript.Extensibility, invoke the provider directly, and service compiled callbacks and tasks through ExternalCallContext. Opaque values retain their SDK payload and immutable field snapshots. Providers taking HostContext receive the process working directory and compiled default functions. The CLI preserves provider metadata from environment loading rather than executing provider factories twice during compilation. No interpreter or ScriptHost participates in generated execution.

Representation and execution

Provider classes may expose public static parameterless FScriptCompiledFunctions and FScriptCompiledTypes methods returning IEnumerable<KeyValuePair<string, MethodInfo>> and IEnumerable<KeyValuePair<string, Type>> respectively. Keys name registered script bindings. Compiler-only discovery links reachable declarations to exact typed CLR methods/types, overriding the legacy SDK adapter for those bindings. Metadata factories and unused bindings do not become runtime roots. This optional BCL contract preserves the frozen SDK registration records. Typed entry points must preserve the registered semantics and use immutable or snapshot-compatible external representations; unsupported shapes can use explicit adapters.

Typed CLR parameters may accept unary delegates. The compiler evaluates the script callback once and emits a typed delegate adapter, recursively converting its argument and result. This includes callbacks that return another delegate, such as Func<Window, Action<double>>; Action<T> discards the script unit result. Unsupported delegate arities report FSC6006. Script data records passed to CLR methods use a public constructor whose parameter names match the script fields, independent of field order. Nested records and supported collections are converted recursively, without runtime reflection or SDK dispatch. Constructor mismatches report FSC6007.

CLR type reachability is computed from retained expression/function signatures, patterns, constructors, and type tokens, including recursive references. Unused imported types are removed before resolving their native metadata. Emitted dependency copying starts at the PE's actual assembly references and follows the supplied adjacent managed closure; simply loading a provider during frontend registration does not retain its assembly.

Compiled Env.ScriptName contains the running assembly or executable filename; Env.Arguments contains its command-line arguments. The BASIC sample uses that name in its usage message, so compiled output does not advertise an interpreter invocation.

Closed exported bindings are specialization roots even when a library has no program entry. An export with an open polymorphic signature receives FSC6009; it cannot silently disappear during pruning. A known closure is devirtualized only when its captured values are available in the caller's lexical scope. Otherwise its delegate environment is preserved.

Scalars use long, double, bool, and string. Functions use typed Func<A,B> delegates. Lists use arrays, options use typed value structs, maps use sorted immutable entry arrays, tuples and records use generated immutable records, and unions use enums or sealed typed cases. Recursive records/unions use CLR references. Collection operations allocate changed values and retain source order. Structural equality and result formatting are generated per semantic type, including recursive aggregate equality and non-equality of functions/tasks.

Generated console entry points print FormatResult(Run()) by default. --no-result suppresses this display for C# projects, ordinary assemblies, single-file bundles, and AOT output. Before specialization, LirResultPolicy.discard replaces the entry value with unit while retaining any observable entry evaluation; unused pure declarations/functions can then be pruned. In discard mode Run returns unit. Exports remain explicit roots. Program effects and failures remain intact. Entry points return the execution-local ExitCode, defaulting to zero, rather than interpreting the program value as a status. Env.setExitCode performs an atomic validated write in the generated program; it does not mutate the host process status when the assembly is embedded. It accepts 0..255, uses last-completed-write-wins semantics, and is always observable. Unhandled failures override selected successful completion.

Match targets are evaluated once. Typed checks/projections, repeated bindings, guards, and deterministic ordered-map selection follow the LIR pattern plans. An unbound variable key selects the first remaining entry; other key/value patterns select their first local match. Selection is committed before later clauses or remainder checks. Failures are ordinary runtime failures. Top-level initialization executes once and in source order through a lazy initializer outside CLR type initialization, allowing spawned tasks to call compiled functions without a type-initialization deadlock.

Output and verification

DotNetExecutableOptions contains the explicit portable .NET RuntimeIdentifier, an AheadOfTime flag, and a MinimizeSize flag. generateExecutable accepts console output only. AheadOfTime=false produces a self-contained single-file bundle; AheadOfTime=true uses Native AOT. Both return exactly one executable (.exe for Windows targets). The CLI enables Unix execute permissions when writing Unix targets. DLL generation and the other backends do not invoke SDK publishing.

CLI compile options may come from the selected manifest's [compile] table. The CLI merges built-in defaults, manifest defaults, and explicit positive/negative flags before choosing a backend; compiler APIs remain explicit and do not discover manifests. See compile defaults for precedence and output directories. The default backend is .NET assembly output, and --target/--output are optional.

The CLI forms are --target dotnet --single-file [--runtime <rid>] and --target dotnet --aot [--runtime <rid>]. Omitting --runtime selects the current .NET host runtime identifier (System.Runtime.InteropServices.RuntimeInformation.RuntimeIdentifier). Explicit --runtime options without a value, runtime identifiers without a publishing mode, and executable flags on other targets are rejected. The backend API retains an explicit runtime identifier. Runtime identifiers use .NET spelling, including osx-arm64, linux-x64, and win-x64. Unsupported identifiers produce FSN7101 with the supplied value, supported identifiers, and a correction for recognized macOS aliases before invoking the publishing SDK. With source invocation, application arguments must follow dotnet run --; options consumed by dotnet run cannot be validated by FScript. Native AOT cross-OS compilation is rejected; cross-architecture support depends on the installed toolchain.

CLI compilation reports its target and requested runtime, selected manifest when present, code-generation phase, and SDK publishing phase on stdout. The publishing message is emitted only after managed generation and AOT compatibility checks succeed, immediately before SDK publishing starts. Success is reported after writing artifacts and permissions, with total elapsed time and the primary output path; failures keep diagnostics on stderr and emit no success message. Backend embedding APIs remain silent by default; generateExecutableWithResultOutputAndProgress accepts an explicit progress callback for callers that want publishing notifications.

Publishing stages the optimized PE and managed dependency closure in an isolated temporary SDK project. Its CoreCompile placeholder is replaced with the optimized PE before publish targets consume it. Single-file publishing embeds the runtime and native runtime assets for extraction. Native AOT consumes the same PE, with SDK trimming and native compilation. Managed bundles enable trimming for static bindings and disable it for reachable legacy SDK dispatch. Trimmer diagnostics report individual warnings rather than assembly-only summaries. Only IL2040 (missing resources requested for removal by dependency linker descriptors, including optional FSharp.Core compiler metadata) is suppressed; reflection, trimming and AOT compatibility warnings remain errors. Remaining unbundled publish assets cause a diagnostic rather than incomplete output. Debug-symbol files are optional and are not returned.

Native AOT always uses OptimizationPreference=Size, keeping standard culture support, framework exception messages, stack traces, and EventSource by default. The optional --size-optimized flag requires --target dotnet --aot and sets MinimizeSize=true in the backend options. It enables invariant globalization and framework resource-key errors, and disables runtime stack traces and EventSource. Provider and FScript error messages remain intact. Providers requiring named cultures or these diagnostic facilities must use ordinary --aot. Reflection analysis and trimming/AOT warnings remain enabled in both modes. MinimizeSize=true with AheadOfTime=false produces FSN7101 before generation or publishing.

Executable publishing defines boolean FScript.NativeAssets.<rid> feature switches for every supported portable runtime identifier, true only for the requested identifier. Providers may opt in through embedded ILLink.Substitutions.xml resource-removal rules conditioned on their runtime's switch being false. Missing switches must retain assets, so normal DLL generation and interpreted provider loading remain portable. The compiler never guesses which arbitrary provider resources are disposable. The FFI renderer uses this contract to retain its GLFW license and selected native library while dropping other platforms' libraries in trimmed single-file and AOT output. Untrimmed legacy single-file output retains all resources.

Publishing requires a .NET 10 SDK on PATH and, for Native AOT, the platform native toolchain and compatible dependencies. SDK restore may need network access. FSN7101 reports invalid publishing options, FSN7102 reports an unavailable SDK command, FSN7103 reports SDK publishing failures/timeouts or incomplete bundling, and FSN7104 reports Native AOT cross-OS requests. The compiler captures SDK diagnostics and cleans its temporary project. Existing unrelated output-directory files are preserved.

C# output includes Program.cs, Runtime.cs, and Program.csproj. .NET output includes the assembly and, for console applications, a runtime configuration. External assembly references are explicit. Source generation does not invoke an installed compiler. Ordinary assembly generation requires the .NET 10 runtime but not an installed SDK; Pinned, architecture-neutral Roslyn NuGet assemblies ship with FScript. Build-host SDK Roslyn binaries must not enter release packages because their ReadyToRun code can target the build host rather than the CLI's target. Release CI tests the actual macOS archive on an ARM64 macOS runner before publication. Running the compiler from source through dotnet run and SDK publishing do require the SDK. Assembly generation uses Roslyn in-process with release optimization, deterministic output, unsafe code disabled, and nullable checks enabled. Compiler errors and warnings prevent publication of an output assembly.

Tests load emitted assemblies, JIT every closed generated method, execute representative semantic programs and CLR calls, and build generated C# as an independent project. This checks valid IL and execution behavior without relying on FScript hosting.

Native AOT reports FSN7105 when the optimized PE still contains reachable legacy SdkInterop dispatch. Such calls require typed companions or explicit CLR bindings, allowing individual FFI implementations to be inlined/trimmed without retaining a provider factory. Managed bundle mode continues to support legacy dispatch. Integration tests execute both output modes from a directory containing only the executable, with absent runtime roots, and cover direct typed FFI plus external values with an unused reflection-based sibling.

The standalone BASIC and headless graphics assemblies are also checked with ILVerify against the installed .NET runtime and emitted managed dependencies. C# output is syntax-checked and normalized before publication; generated projects explicitly include compiler-owned source files so stale files cannot enter a later build.

SDK programs additionally include RuntimeSDK.cs and explicit references to the extension SDK and FSharp.Core. Colocated managed dependencies are resolved transitively and copied beside emitted assemblies. Generated programs never load the source file or a serialized IR at runtime. The graphics sample uses static companions for opaque graphics resources, geometry values, and typed callback invocation. Native AOT integration tests compile the real teapot script with an automatically discovered manifest and run the executable from a directory without source or provider files. Its GLFW runtime binaries and license are embedded in the renderer; graphical startup extracts the current platform binary without consulting managed assembly locations.

generateWithResultOutput on both backends and generateExecutableWithResultOutput on the .NET backend accept an explicit Boolean printing policy. Existing generation methods select printing. Native Main returns int; Run, FormatResult, exported bindings, and the public ExitCode property form the generated host surface.

Explicit foreign IL expansion

FScriptInlineFunctions(): IEnumerable<string> is additive compiler-only provider metadata. Names select exact FScriptCompiledFunctions bindings and do not imply Pure. ClrBindings.bindInlineMethod is the equivalent compiler API. The Inline Boolean is serialized in the existing CLR method-link JSON carried by LIR target imports; absent metadata defaults false. No language/Fable/evaluator changes or IR format version bump are required.

Assembly emission inserts unique binding-link markers, preserving distinct opt-in/unmarked aliases of one CLR method. A post-Roslyn Mono.Cecil pass expands eligible marked static managed bodies, remaps arguments/locals/branches/returns, preserves per-invocation initialized locals, folds total integer constants with CLI wrap semantics, removes markers, and prunes proven unused provider assembly scopes. Exact forwarding bodies retarget the call directly. Public helper calls remain calls; no recursive helper expansion is inferred. No unmarked FFI call is transformed. C# generation ignores expansion while retaining typed linkage.

Initial eligibility excludes generic owners/methods, instance methods/constructors, module/type initializers, exception handlers, non-public references, pointer/byref signatures, synchronized/native/runtime bodies, explicit NoInlining, prefixes and unsupported frame instructions, self recursion, and known stack-sensitive reflection. Authors additionally promise call-site independence; arbitrary stack inspection cannot be proved absent transitively. Unsupported reached bodies report FSN7201 with binding span; rewrite/metadata failures report FSN7202. Unused bodies are not validated. Rewriting is deterministic and does not alter SDK registration records or interpreter execution.

Tests cover granularity, ordered effects, void and value returns, branches/loops, retained failures, private/EH rejection, dead bindings, artifact round trips, deterministic bytes, dependency pruning, and unchanged C# output. CLR JIT preparation runs for all closed generated methods; a representative mixed-body assembly is additionally checked by ILVerify when installed.

Foreign expansion profitability

Inline is permission rather than unconditional expansion. The initial policy uses original callee IL bytes (excluding metadata): up to 16 is tiny, 17–100 discretionary, and above 100 remains an exact typed call. Ordinary candidates with more than 32 parameters or locals are also retained. These screens follow .NET 10 RyuJIT's normal constants, not its dynamic profitability model; F# expression weights are not treated as byte counts.

A trial clones the caller, including EH boundaries and locals, performs expansion and cleanup, then measures compact encoded IL. Per-site positive growth is capped at 32 bytes for tiny bodies and 16 for discretionary bodies. Cumulative growth is capped at max(32, min(256, originalCallerBytes)). Failed trials restore the original body and retarget the marker directly to the original method; no temporary state leaks. Exact same-signature forwarders are collapsed without duplicating a body or consuming growth budget. Pinned locals are not eligible for copying. Size/profitability skips are successful compilation, while selected unsupported bodies retain FSN7201 diagnostics. Tests cover a retained >100-byte body, an unprofitable medium loop, an expanded tiny loop, and controlled single-caller cumulative growth.

Marker discovery includes generated closure/display classes, so escaped and partially applied FFI bindings receive the same expansion and budgets as saturated calls. Before deleting markers the pass rejects any unsupported remaining method-token/function-pointer reference, preventing dangling metadata or invalid IL. Regression tests execute both escaping one-argument and captured two-argument bindings.

Capture cleanup after tail-loop lowering

The assembly backend specializes known captured functions before native lowering. A local partial function whose implementation and captures are known can become direct arithmetic in its consumer, including a recursive loop. Actual captures remain explicit; separate invocations retain separate immutable environments.

After tail calls become loops, the backend removes unused private native parameters and their inert arguments, then removes unreachable local closure cycles and capture snapshots. These passes iterate to a fixed point before member pruning. Delegate method-group references retain their original signatures. Calls and external property reads with possible effects or failures remain ordered. Escaping functions and unknown callbacks can still require delegates. This cleanup does not change interpreter execution or serialized LIR.

Temporary records and representation conversions

Both compiled targets scalar-replace internal record temporaries used only by field projections, updates, or compatible width conversions. Field expressions keep their original evaluation order and execute once, including unused fields that perform I/O or fail. A projection through a compatible internal record conversion can read the original field directly. A record returned under a different internal record signature can be constructed in the destination representation.

The .NET backend repeats this cleanup after known-call reduction and reachability pruning, so removing an unused closure can expose further temporary records. Records that escape through unknown consumers or captures remain ordinary values. Imported record constructors/getters and nested field adaptations are conservative boundaries; this pass does not treat them as plain field copies. Record scalar replacement alone does not fuse collection pipelines or eliminate their materialized result collections; the assembly-only fusion pass described below can remove intermediate arrays and expose further record elimination.

Collection representation propagation

The assembly backend carries a consistent internal record element representation through known List.map, List.filter, and List.fold consumers. Single-use lifted callbacks can receive the producer's representation directly; their original field semantics are preserved by explicit conversions that subsequent scalar replacement can remove. A private collection binding can share that representation between compatible consumers without converting the complete array.

Public returns, escaping/captured collection bindings, imported records, and unknown callbacks retain conversion boundaries. Callback evaluations and stage order are unchanged. This pass does not require callbacks to be pure because it changes representations rather than interleaving their execution. The C# source target and serialized IR retain their existing representations.

Pure collection pipeline fusion

The assembly backend fuses single-consumer List.map and List.filter chains ending in List.fold when the complete region is proven pure and total. It evaluates callback setup once, constructs a bounded tail-recursive loop in private LIR, and reuses native call reduction, scalar replacement, and tail-loop lowering. Intermediate arrays disappear; nonescaping mapped records can disappear as well. Shared or escaping producer bindings remain materialized.

Range sources can feed the loop directly, including descending ranges. The last endpoint is processed before deciding whether to increment, avoiding overflow at signed integer limits. Other list sources are evaluated once and indexed with a checked, typed option decision. Unknown calls, I/O, tasks, possible failures and incomplete matches prevent fusion; their existing stage order remains intact. The shared effect analysis proves unguarded exhaustive union matches and irrefutable typed tuple/record patterns total, while retaining guards and repeated-binding checks as possible failures. No public IR node or serialization change is needed.

Native tuple temporaries

The assembly backend replaces local tuple constructors used only for irrefutable destructuring with independent item bindings. Every item still evaluates once in source order. Guards, repeated bindings, nested coercions and escaping uses retain normal pattern decisions and values.

Small tuples of up to four simple scalar/reference fields use generated readonly value types in the .NET target, allowing native fold accumulators and method arguments to avoid heap carriers. Larger or nested/opaque layouts retain reference carriers. The source target keeps its existing tuple representation. Both targets preserve the semantic tuple type and dedicated structural equality/formatting; no CLR record equality or boxing is required for typed native tuple operations. Value-type constructor newobj instructions do not themselves imply heap allocation, so allocation measurements remain the check for per-iteration cost.

Private list-tail views

Native method parameters, local list values and list-pattern bindings can use a readonly view of an owned array plus offset and length. List.tail and cons-pattern remainders then advance a view instead of copying every suffix. Native tail-call reassignment preserves that representation; known collection loops use a value-type enumerator or index the view directly. Equality and formatting have typed view consumers. Native cons/append helpers copy spans directly into an exactly sized result buffer; appending a singleton avoids a separate one-element array, and appending an empty list can reuse the immutable view. Readonly collection interfaces retain efficient counted access for fallback BCL consumers.

Exports, ordinary delegates, aggregate fields and FFI keep their array representation. Partial views materialize at those array boundaries; a complete view can reuse its already owned array. Escaping closure captures take an array snapshot when the closure is created, rather than repeatedly materializing on invocation or retaining an unrelated array prefix. Existing array-returning native methods keep their ownership boundary. Some fallback enumerable consumers can still allocate iterator objects.

The view runtime is embedded only for assembly generation and pruned when unused. Source projects retain arrays. Empty List.tail raises the documented evaluation error in both compiled targets; a nonempty singleton tail is an empty list. Source reachability follows implicit conversion operators and converted types so all required view/array adapters survive pruning. There is no public view type in the FScript language or IR artifact.

Exact-size compiled filters

Materialized List.filter operations in both compiled targets record selection decisions in compact 32-bit masks, count accepted elements, then allocate and fill an exactly sized result array. Inputs of up to 32 elements keep their mask in a scalar; larger inputs allocate one bit per element rounded to mask words. Empty results reuse the empty array. Workers build masks a word at a time and copy selected elements from a local readonly span, without growing element buffers. The mask strategy preserves semantics for effectful predicates and does not assume they can be repeated. Known callbacks retain direct typed calls; unknown callbacks use a typed runtime helper.

Predicates run exactly once per element in source order, including effectful predicates. A failing predicate stops the operation immediately. The second traversal only copies accepted values and does not invoke the predicate again. These are private backend bit operations; no bit-operation syntax or public IR change is introduced. Pure pipeline fusion can still eliminate materialized filters entirely.