From 69dbf6a0fd3fb477e6bdf7302115f032647f8a50 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 16:17:11 +0200 Subject: [PATCH 001/133] docs(compiler): define semantic baseline and IR architecture --- CHANGELOG.md | 1 + docs/architecture/README.md | 1 + docs/architecture/typed-ir-and-backends.md | 41 ++++++++++++++++++++++ docs/specs/README.md | 1 + docs/specs/compiler-semantics-baseline.md | 41 ++++++++++++++++++++++ 5 files changed, 85 insertions(+) create mode 100644 docs/architecture/typed-ir-and-backends.md create mode 100644 docs/specs/compiler-semantics-baseline.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 59a676f..b21e2a6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Documented the semantic baseline and architecture for typed IR, executable LIR, and compiled backends. - Rewrote the website around concrete FScript behavior and removed vague product slogans. - Fixed oversized homepage headings to wrap in balanced lines across desktop widths. diff --git a/docs/architecture/README.md b/docs/architecture/README.md index 5ad77a4..2a147d2 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -4,3 +4,4 @@ How the repository and assemblies are organized. - Assemblies and roles: [`assemblies-and-roles.md`](./assemblies-and-roles.md) - REPL architecture: [`repl-architecture.md`](./repl-architecture.md) +- Typed IR and backends: [`typed-ir-and-backends.md`](./typed-ir-and-backends.md) diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md new file mode 100644 index 0000000..6c18cd4 --- /dev/null +++ b/docs/architecture/typed-ir-and-backends.md @@ -0,0 +1,41 @@ +# Typed IR and Backend Architecture + +## Pipeline + +```text +source -> parser/includes -> inference + resolved typed tree -> HIR -> LIR + | | + | +-> LIR interpreter + +--------> serialization + +-> Go source backend +``` + +`FScript.Language` remains the parser, type-system, legacy evaluator, and Fable-compatible package. It emits a resolved typed tree in the same inference pass as its compatibility `TypedProgram`. + +`FScript.Compiler` is a reusable, initially non-packaged .NET assembly. It owns HIR, HIR validation, lowering, LIR, LIR validation, artifact serialization, LIR interpretation, and target backends. It depends on `FScript.Language`; the language assembly never depends on the compiler. + +## IR boundaries + +HIR is language-oriented and contains resolved artifact-local symbol identities, complete semantic types and schemes, source locations, declaration attributes, and explicit evidence for named/structural record conversions. It contains no Go types. + +LIR is executable but not a Go AST. It uses lifted functions, explicit captures and closure creation, resolved primitive operations, aggregate construction/projection, and structured control flow. Pattern matching becomes deterministic tests and projections before reaching a backend. Semantic types may remain alongside selected runtime representations for validation and diagnostics. + +Serialized `.fsir` files contain one closed resolved program plus explicit foreign linkage declarations. JSON format versions are internal and may be rejected rather than upgraded. + +## Representation policy + +- Primitive values, declared records, tuple shapes, and declared unions receive concrete representations where possible. +- Polymorphic lists, options, maps, and the universal curried closure calling convention may carry erased payloads. +- Erasure is introduced during HIR-to-LIR lowering, never during frontend inference. +- Structural equality and record conversions are explicit LIR operations/helpers; Go `==` and Go nominal assignability never define FScript semantics. +- LIR remains structured and non-SSA until a concrete optimization requirement justifies a control-flow IR. + +## Runtime and foreign calls + +Compiler bindings classify names as portable intrinsics, backend runtime primitives, target imports, or unsupported host externs. This metadata is distinct from executable .NET `ExternalFunction` implementations. + +The LIR interpreter rejects reachable target imports it cannot supply. The Go backend initially rejects unsupported host externs and tasks with backend diagnostics. Imports describe complete ordinary FScript declarations plus target linkage and do not create a second foreign type system. + +## Previous experiment + +The `feature/in-memory-execution` branch is useful only as evidence about closures, invocation, and host integration. Its dynamic-method/in-memory assembly backend is not an IR layer and is neither copied nor referenced by the new compiler architecture. diff --git a/docs/specs/README.md b/docs/specs/README.md index 4748f49..284cf8b 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -4,6 +4,7 @@ Normative behavior for the language, runtime surface, hosting model, and editor/ ## Language and type system +- Compiler semantic baseline: [`compiler-semantics-baseline.md`](./compiler-semantics-baseline.md) - Syntax and indentation: [`syntax-and-indentation.md`](./syntax-and-indentation.md) - Supported types: [`supported-types.md`](./supported-types.md) - Function annotations: [`function-annotations.md`](./function-annotations.md) diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md new file mode 100644 index 0000000..d6359e4 --- /dev/null +++ b/docs/specs/compiler-semantics-baseline.md @@ -0,0 +1,41 @@ +# Compiler Semantic Baseline + +## Purpose + +This document records the behavior that the typed IR, LIR interpreter, and compiled backends must preserve. The current type inference and evaluator are the reference until cross-runtime conformance replaces that dependency. + +## Frontend and execution + +- Includes are resolved into one flattened program before inference. Alias-qualified declarations are rewritten by the include resolver; unresolved `SImport` nodes are rejected by inference. +- Hindley-Milner inference generalizes top-level and local let bindings after applying substitutions. Recursive bindings are inferred monomorphically inside their recursive group and generalized afterward. +- The active typed program stores top-level types but retains parser expressions in bodies. Compiler lowering therefore requires a new resolved, recursively typed result emitted by the inference pass; types cannot safely be reconstructed afterward. +- Top-level statements execute in source order. Every non-final expression must have type `unit`; the final expression is the program result. +- Functions are curried. Multi-argument declarations are nested lambdas, application is one argument at a time, and closures capture their lexical environment. Single and mutual recursive function groups are supported. + +## Types and values + +- Primitive runtime values are unit, signed 64-bit integer, IEEE-754 double, Boolean, and string. +- Tuples and lists preserve element order. Options distinguish `None` from `Some value`. +- Maps currently accept string keys through inference and have deterministic ordered-map behavior. Runtime support for integer map keys is legacy representation detail, not permission for newly compiled programs. +- Declared records retain declaration identity for named annotations. Structural record annotations retain field-shape compatibility and may accept additional fields where current inference permits it. A backend must insert representation conversions rather than change compatibility rules. +- Discriminated unions retain their declared type name, case name, and optional payload. Recursive types require `type rec`; mutually recursive type-definition graphs are rejected. +- `typeof` produces an FScript type token and `nameof` produces the resolved source name. + +## Expressions, patterns, and equality + +- Conditionals and match guards require Boolean values and evaluate only the selected branch. +- Match clauses are tried in source order. A guard runs only after its pattern binds successfully. Failure to find a clause is a runtime error. +- Patterns cover literals, variables, wildcards, lists, tuples, records, ordered maps, options, union cases, and type-reference tests. Let-pattern failure is a runtime error. +- Record updates evaluate the base once and then updates in source order. Ranges, loops, interpolation, indexing, raise, append, and cons preserve the evaluator's existing evaluation order and failures. +- Equality is structural for primitives, tuples, records, maps, options, lists, unions, and type tokens, recursively. Functions, tasks, constructors, and extern handles are not structurally equal. Backend host-language comparability must not leak into this behavior. + +## Builtins, externs, attributes, and tasks + +- Scalar conversions, string helpers, `List`, `Option`, and `Map` functions are protected builtin symbols with explicit polymorphic schemes. They are initial compiler/runtime intrinsics even where a future implementation can compile their FScript source. +- `Console.writeLine`, `Console.readLine`, and the CLI `Env` value are candidate portable runtime primitives. Filesystem, regex, hashing, GUID, JSON, and XML functions remain host externs initially. +- `Task.spawn` and `Task.await` are host externs with tracked-task semantics, background error propagation, and an error for unawaited tasks. HIR may represent them before LIR interpreters or backends support them. +- `[]` is declaration metadata used by embedding hosts and does not change inference. Backend `Import` metadata is separate and will not weaken the declared FScript type. + +## Diagnostics + +Parse, type, and evaluation failures currently use source spans and exceptions. Compiler entry points translate expected failures into structured stage diagnostics while leaving existing embedding APIs behavior-compatible. From 06bcc8755a0440d4d2d561191fead6b340905164 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 16:21:54 +0200 Subject: [PATCH 002/133] refactor(language): expose resolved expression types --- CHANGELOG.md | 1 + src/FScript.Language/Library.fs | 2 + src/FScript.Language/TypeInfer.fs | 154 ++++++++++++------ .../FScript.Language.Tests.fsproj | 1 + .../ResolvedInferenceTests.fs | 31 ++++ 5 files changed, 141 insertions(+), 48 deletions(-) create mode 100644 tests/FScript.Language.Tests/ResolvedInferenceTests.fs diff --git a/CHANGELOG.md b/CHANGELOG.md index b21e2a6..8f61934 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added a compatibility-preserving resolved inference result with expression-level type information for compiler frontends. - Documented the semantic baseline and architecture for typed IR, executable LIR, and compiled backends. - Rewrote the website around concrete FScript behavior and removed vague product slogans. - Fixed oversized homepage headings to wrap in balanced lines across desktop widths. diff --git a/src/FScript.Language/Library.fs b/src/FScript.Language/Library.fs index 8d4ec3b..b0c77e6 100644 --- a/src/FScript.Language/Library.fs +++ b/src/FScript.Language/Library.fs @@ -8,6 +8,8 @@ module FScript = let parseSourceWithIncludesResolver = IncludeResolver.parseProgramFromSourceWithIncludesResolver let infer = TypeInfer.inferProgram let inferWithExterns = TypeInfer.inferProgramWithExterns + let inferResolved = TypeInfer.inferResolvedProgram + let inferResolvedWithExterns = TypeInfer.inferResolvedProgramWithExterns let eval = Eval.evalProgram let evalWithExterns = Eval.evalProgramWithExterns diff --git a/src/FScript.Language/TypeInfer.fs b/src/FScript.Language/TypeInfer.fs index 7c8dce1..4890ce9 100644 --- a/src/FScript.Language/TypeInfer.fs +++ b/src/FScript.Language/TypeInfer.fs @@ -159,6 +159,16 @@ module TypeInfer = Span: Span Type: Type } + type ResolvedExpressionType = + { Expr: Expr + Type: Type + Span: Span } + + type ResolvedSymbolReference = + { SymbolId: int + Name: string + Span: Span } + type private LocalVariableTypeCapture = { Name: string Span: Span @@ -171,13 +181,20 @@ module TypeInfer = | TSExpr of TypedExpr type TypedProgram = TypedStmt list + type ResolvedProgram = + { TypedProgram: TypedProgram + ExpressionTypes: ResolvedExpressionType list + SymbolReferences: ResolvedSymbolReference list } + type private InferenceTelemetry = { mutable ScopeStack: Set list - LocalVariableTypes: ResizeArray } + LocalVariableTypes: ResizeArray + ExpressionTypes: ResizeArray } let private createTelemetry () = { ScopeStack = [ Set.empty ] - LocalVariableTypes = ResizeArray() } + LocalVariableTypes = ResizeArray() + ExpressionTypes = ResizeArray() } let private capturedLocalVariableTypes (telemetry: InferenceTelemetry option) : LocalVariableTypeInfo list = match telemetry with @@ -243,6 +260,14 @@ module TypeInfer = let entry = t.LocalVariableTypes[i] entry.Type <- applyType subst entry.Type + let private applySubstToCapturedExpressions (telemetry: InferenceTelemetry option) (startIndex: int) (subst: Subst) = + match telemetry with + | Some t -> + for i = startIndex to t.ExpressionTypes.Count - 1 do + let item = t.ExpressionTypes.[i] + t.ExpressionTypes.[i] <- { item with Type = applyType subst item.Type } + | None -> () + let private currentTelemetryCaptureCount (telemetry: InferenceTelemetry option) = match telemetry with | Some t -> t.LocalVariableTypes.Count @@ -252,7 +277,16 @@ module TypeInfer = { UnionName: string Payload: Type option } - let private asTyped expr t = { Expr = expr; Type = t; Span = Ast.spanOfExpr expr } + let private asTyped (telemetry: InferenceTelemetry option) expr t : TypedExpr = + let span = Ast.spanOfExpr expr + match telemetry with + | Some capture -> + capture.ExpressionTypes.Add + { Expr = expr + Type = t + Span = span } + | None -> () + { Expr = expr; Type = t; Span = span } let private builtinType (name: string) : Type option = match name with @@ -559,7 +593,7 @@ module TypeInfer = let rec private inferExpr (telemetry: InferenceTelemetry option) (typeDefs: Map) (constructors: Map) (env: Map) (expr: Expr) : Subst * Type * TypedExpr = match expr with | EUnit _ -> - emptySubst, TUnit, asTyped expr TUnit + emptySubst, TUnit, asTyped telemetry expr TUnit | ELiteral (lit, _) -> let t = match lit with @@ -567,25 +601,25 @@ module TypeInfer = | LFloat _ -> TFloat | LBool _ -> TBool | LString _ -> TString - emptySubst, t, asTyped expr t + emptySubst, t, asTyped telemetry expr t | EVar (name, span) -> match env |> Map.tryFind name with | Some scheme -> let t = instantiate scheme recordLocalVariableUse telemetry name span t - emptySubst, t, asTyped expr t + emptySubst, t, asTyped telemetry expr t | None -> raise (TypeException { Message = sprintf "Unbound variable '%s'" name; Span = span }) | EParen (inner, _) -> let s, t, _ = inferExpr telemetry typeDefs constructors env inner - s, t, asTyped expr t + s, t, asTyped telemetry expr t | ETypeOf (name, span) -> if typeDefs.ContainsKey name then - emptySubst, TTypeToken, asTyped expr TTypeToken + emptySubst, TTypeToken, asTyped telemetry expr TTypeToken else raise (TypeException { Message = sprintf "Unknown type '%s'" name; Span = span }) | ENameOf (name, span) -> if env.ContainsKey name then - emptySubst, TString, asTyped expr TString + emptySubst, TString, asTyped telemetry expr TString else raise (TypeException { Message = sprintf "Unbound variable '%s'" name; Span = span }) | EInterpolatedString (parts, span) -> @@ -596,7 +630,7 @@ module TypeInfer = | IPExpr pexpr -> let s1, _, _ = inferExpr telemetry typeDefs constructors (applyEnv sAcc env) pexpr sAcc <- compose s1 sAcc - sAcc, TString, asTyped expr TString + sAcc, TString, asTyped telemetry expr TString | ELambda (param, body, _) -> let tv = match param.Annotation with @@ -609,7 +643,7 @@ module TypeInfer = inferExpr telemetry typeDefs constructors env' body) let tArg = applyType s1 tv let tFun = TFun (tArg, tBody) - s1, tFun, asTyped expr tFun + s1, tFun, asTyped telemetry expr tFun | EApply (fn, arg, span) -> let s1, t1, _ = inferExpr telemetry typeDefs constructors env fn let env2 = applyEnv s1 env @@ -618,7 +652,7 @@ module TypeInfer = let s3 = unify typeDefs (applyType s2 t1) (TFun (t2, tv)) span let s = compose s3 (compose s2 s1) let tRes = applyType s tv - s, tRes, asTyped expr tRes + s, tRes, asTyped telemetry expr tRes | EIf (cond, tExpr, fExpr, span) -> let s1, tCond, _ = inferExpr telemetry typeDefs constructors env cond let sBool = unify typeDefs tCond TBool span @@ -629,13 +663,13 @@ module TypeInfer = let s4 = unify typeDefs (applyType s3 tThen) tElse span let s = compose s4 (compose s3 (compose s2 (compose sBool s1))) let tRes = applyType s tElse - s, tRes, asTyped expr tRes + s, tRes, asTyped telemetry expr tRes | ERaise (value, span) -> let s1, t1, _ = inferExpr telemetry typeDefs constructors env value let s2 = unify typeDefs (applyType s1 t1) TString span let s = compose s2 s1 let tRes = Types.freshVar() - s, tRes, asTyped expr tRes + s, tRes, asTyped telemetry expr tRes | EFor (name, source, body, span) -> let s1, tSource, _ = inferExpr telemetry typeDefs constructors env source let tv = Types.freshVar() @@ -650,7 +684,7 @@ module TypeInfer = inferExpr telemetry typeDefs constructors envBody body) let sBodyUnit = unify typeDefs (applyType s2 tBody) TUnit (Ast.spanOfExpr body) let s = compose sBodyUnit (compose s2 sSource) - s, TUnit, asTyped expr TUnit + s, TUnit, asTyped telemetry expr TUnit | ELet (name, value, body, isRec, returnAnnotation, span) -> if isRec then let tv = Types.freshVar() @@ -674,7 +708,7 @@ module TypeInfer = withLocalBindings telemetry [ name ] (fun () -> inferExpr telemetry typeDefs constructors env2 body) let s = compose s3 sValue - s, tBody, asTyped expr tBody + s, tBody, asTyped telemetry expr tBody else let s1, t1, _ = inferExpr telemetry typeDefs constructors env value let sDiscard = @@ -697,7 +731,7 @@ module TypeInfer = withLocalBindings telemetry [ name ] (fun () -> inferExpr telemetry typeDefs constructors env2 body) let s = compose s2 sValue - s, t2, asTyped expr t2 + s, t2, asTyped telemetry expr t2 | ELetPattern (pat, value, body, span) -> let startIndex = currentTelemetryCaptureCount telemetry let s1, tValue, _ = inferExpr telemetry typeDefs constructors env value @@ -717,7 +751,7 @@ module TypeInfer = withLocalBindings telemetry boundNames (fun () -> inferExpr telemetry typeDefs constructors env2 body) let s = compose sBody sValue - s, tBody, asTyped expr tBody + s, tBody, asTyped telemetry expr tBody | ELetRecGroup (bindings, body, span) -> let foldedBindings = bindings @@ -770,7 +804,7 @@ module TypeInfer = withLocalBindings telemetry recursiveNames (fun () -> inferExpr telemetry typeDefs constructors envBody body) let s = compose sBody sRec - s, tBody, asTyped expr tBody + s, tBody, asTyped telemetry expr tBody | EMatch (scrutinee, cases, span) -> let s1, tScrut, _ = inferExpr telemetry typeDefs constructors env scrutinee let mutable sAcc = s1 @@ -829,7 +863,7 @@ module TypeInfer = Some (applyType sEq tPrev) sAcc <- compose sBody (compose sPat sAcc) let tRes = resultTypeOpt |> Option.defaultValue (Types.freshVar()) - sAcc, tRes, asTyped expr tRes + sAcc, tRes, asTyped telemetry expr tRes | EList (items, span) -> let tv = Types.freshVar() let mutable sAcc = emptySubst @@ -838,7 +872,7 @@ module TypeInfer = let s2 = unify typeDefs (applyType s1 tv) t1 span sAcc <- compose s2 (compose s1 sAcc) let tRes = TList (applyType sAcc tv) - sAcc, tRes, asTyped expr tRes + sAcc, tRes, asTyped telemetry expr tRes | ERange (startExpr, endExpr, span) -> let s1, tStart, _ = inferExpr telemetry typeDefs constructors env startExpr let env2 = applyEnv s1 env @@ -847,7 +881,7 @@ module TypeInfer = let s4 = unify typeDefs (applyType s3 (applyType s2 tEnd)) TInt span let s = compose s4 (compose s3 (compose s2 s1)) let tRes = TList TInt - s, tRes, asTyped expr tRes + s, tRes, asTyped telemetry expr tRes | ETuple (items, _) -> let mutable sAcc = emptySubst let mutable inferred : Type list = [] @@ -856,7 +890,7 @@ module TypeInfer = sAcc <- compose s1 sAcc inferred <- inferred @ [ applyType sAcc t1 ] let tRes = TTuple inferred - sAcc, tRes, asTyped expr tRes + sAcc, tRes, asTyped telemetry expr tRes | ERecord (fields, span) -> let mutable sAcc = emptySubst let mutable inferred : Map = Map.empty @@ -869,7 +903,7 @@ module TypeInfer = let sRecord = unify typeDefs (applyType sAcc (TRecord inferred)) expected span let s = compose sRecord sAcc let tRes = applyType s expected - s, tRes, asTyped expr tRes + s, tRes, asTyped telemetry expr tRes | EStructuralRecord (fields, _) -> let mutable sAcc = emptySubst let mutable inferred : Map = Map.empty @@ -878,7 +912,7 @@ module TypeInfer = sAcc <- compose s1 sAcc inferred <- inferred.Add(name, applyType sAcc t1) let tRes = TRecord inferred - sAcc, tRes, asTyped expr tRes + sAcc, tRes, asTyped telemetry expr tRes | EMap (entries, span) -> let ensureSupportedMapKeyType (t: Type) = match t with @@ -914,7 +948,7 @@ module TypeInfer = ensureSupportedMapKeyType (applyType sAcc keyType) let tRes = TMap (applyType sAcc keyType, applyType sAcc valueType) - sAcc, tRes, asTyped expr tRes + sAcc, tRes, asTyped telemetry expr tRes | ERecordUpdate (baseExpr, updates, span) -> let sBase, tBase, _ = inferExpr telemetry typeDefs constructors env baseExpr let baseFields = @@ -938,7 +972,7 @@ module TypeInfer = sAcc <- compose sEq (compose sField sAcc) let resultFields = baseFields |> Map.map (fun _ t -> applyType sAcc t) let tRes = TRecord resultFields - sAcc, tRes, asTyped expr tRes + sAcc, tRes, asTyped telemetry expr tRes | EStructuralRecordUpdate (baseExpr, updates, span) -> let sBase, tBase, _ = inferExpr telemetry typeDefs constructors env baseExpr let baseFields = @@ -963,7 +997,7 @@ module TypeInfer = sAcc <- compose sEq (compose sField sAcc) resultFields <- resultFields |> Map.add name (applyType sAcc tFieldApplied) let tRes = TRecord (resultFields |> Map.map (fun _ t -> applyType sAcc t)) - sAcc, tRes, asTyped expr tRes + sAcc, tRes, asTyped telemetry expr tRes | EFieldGet (target, fieldName, span) -> let inferRecordField s1 tTarget = match applyType s1 tTarget with @@ -991,15 +1025,15 @@ module TypeInfer = match env.TryFind qualifiedName with | Some scheme -> let t = instantiate scheme - emptySubst, t, asTyped expr t + emptySubst, t, asTyped telemetry expr t | None -> let s1, tTarget, _ = inferExpr telemetry typeDefs constructors env target let sField, tField = inferRecordField s1 tTarget - sField, tField, asTyped expr tField + sField, tField, asTyped telemetry expr tField | _ -> let s1, tTarget, _ = inferExpr telemetry typeDefs constructors env target let sField, tField = inferRecordField s1 tTarget - sField, tField, asTyped expr tField + sField, tField, asTyped telemetry expr tField | EIndexGet (target, keyExpr, span) -> let s1, tTarget, _ = inferExpr telemetry typeDefs constructors env target let env2 = applyEnv s1 env @@ -1011,7 +1045,7 @@ module TypeInfer = let s3 = unify typeDefs keyType TInt span let s = compose s3 (compose s2 s1) let tRes = TOption (applyType s itemType) - s, tRes, asTyped expr tRes + s, tRes, asTyped telemetry expr tRes | TMap (mapKeyType, valueType), _ -> let s3 = unify typeDefs keyType mapKeyType span let s = compose s3 (compose s2 s1) @@ -1021,19 +1055,19 @@ module TypeInfer = | t -> raise (TypeException { Message = $"Map key type must be string, got {Types.typeToString t}"; Span = span }) let tRes = TOption (applyType s valueType) - s, tRes, asTyped expr tRes + s, tRes, asTyped telemetry expr tRes | TVar _, TInt -> let itemType = Types.freshVar() let s3 = unify typeDefs targetType (TList itemType) span let s = compose s3 (compose s2 s1) let tRes = TOption (applyType s itemType) - s, tRes, asTyped expr tRes + s, tRes, asTyped telemetry expr tRes | TVar _, TString -> let valueType = Types.freshVar() let s3 = unify typeDefs targetType (TMap (TString, valueType)) span let s = compose s3 (compose s2 s1) let tRes = TOption (applyType s valueType) - s, tRes, asTyped expr tRes + s, tRes, asTyped telemetry expr tRes | TVar _, TVar _ -> raise (TypeException { Message = "Ambiguous index access: use a list with int index or a map with string key"; Span = span }) | _ -> @@ -1054,7 +1088,7 @@ module TypeInfer = let s3 = unify typeDefs (applyType s2 tRight) (TFun (applyType s2 tLeft, tv)) span let s = compose s3 (compose s2 s1) let tRes = applyType s tv - s, tRes, asTyped expr tRes + s, tRes, asTyped telemetry expr tRes | "::" -> let s1, tHead, _ = inferExpr telemetry typeDefs constructors env a let env2 = applyEnv s1 env @@ -1062,7 +1096,7 @@ module TypeInfer = let s3 = unify typeDefs (applyType s2 tTail) (TList tHead) span let s = compose s3 (compose s2 s1) let tRes = applyType s (TList tHead) - s, tRes, asTyped expr tRes + s, tRes, asTyped telemetry expr tRes | "@" -> let s1, t1, _ = inferExpr telemetry typeDefs constructors env a let env2 = applyEnv s1 env @@ -1072,7 +1106,7 @@ module TypeInfer = let s4 = unify typeDefs (applyType s3 (applyType s2 t2)) (TList (applyType s3 tv)) span let s = compose s4 (compose s3 (compose s2 s1)) let tRes = TList (applyType s tv) - s, tRes, asTyped expr tRes + s, tRes, asTyped telemetry expr tRes | _ -> let s1, t1, _ = inferExpr telemetry typeDefs constructors env a let env2 = applyEnv s1 env @@ -1091,13 +1125,13 @@ module TypeInfer = TBool | _ -> raise (TypeException { Message = sprintf "Unknown operator %s" op; Span = span }) let s = compose s3 (compose s2 s1) - s, resultType, asTyped expr resultType + s, resultType, asTyped telemetry expr resultType | ESome (value, _) -> let s1, t1, _ = inferExpr telemetry typeDefs constructors env value - s1, TOption t1, asTyped expr (TOption t1) + s1, TOption t1, asTyped telemetry expr (TOption t1) | ENone _ -> let tv = Types.freshVar() - emptySubst, TOption tv, asTyped expr (TOption tv) + emptySubst, TOption tv, asTyped telemetry expr (TOption tv) | ECons (head, tail, span) -> let s1, t1, _ = inferExpr telemetry typeDefs constructors env head let env2 = applyEnv s1 env @@ -1105,7 +1139,7 @@ module TypeInfer = let s3 = unify typeDefs (applyType s2 t2) (TList t1) span let s = compose s3 (compose s2 s1) let tRes = applyType s (TList t1) - s, tRes, asTyped expr tRes + s, tRes, asTyped telemetry expr tRes | EAppend (a, b, span) -> let s1, t1, _ = inferExpr telemetry typeDefs constructors env a let env2 = applyEnv s1 env @@ -1115,7 +1149,7 @@ module TypeInfer = let s4 = unify typeDefs (applyType s3 (applyType s2 t2)) (TList (applyType s3 tv)) span let s = compose s4 (compose s3 (compose s2 s1)) let tRes = TList (applyType s tv) - s, tRes, asTyped expr tRes + s, tRes, asTyped telemetry expr tRes let private envFromExterns (externs: ExternalFunction list) : Map = externs @@ -1155,7 +1189,7 @@ module TypeInfer = | SLetRecGroup(bindings, _, _) -> bindings |> List.map (fun (name, _, _, _, _) -> name) | _ -> []) - let private inferProgramWithExternsRawAndLocals (captureLocalVariables: bool) (externs: ExternalFunction list) (program: Program) : TypedProgram * LocalVariableTypeInfo list = + let private inferProgramWithExternsRawAndLocals (captureTelemetry: bool) (externs: ExternalFunction list) (program: Program) : TypedProgram * LocalVariableTypeInfo list * ResolvedExpressionType list = let decls = (BuiltinTypes.typeDefs |> List.map (fun d -> d.Name, d)) @ (program |> List.choose (function | SType d -> Some (d.Name, d) | _ -> None)) @@ -1204,12 +1238,16 @@ module TypeInfer = ||> List.fold (fun acc (name, scheme) -> acc.Add(name, scheme)) let telemetry = - if captureLocalVariables then Some(createTelemetry ()) + if captureTelemetry then Some(createTelemetry ()) else None let typed = ResizeArray() try for i, stmt in program |> List.indexed do + let expressionStartIndex = + match telemetry with + | Some t -> t.ExpressionTypes.Count + | None -> 0 match stmt with | SType def -> typed.Add(TSType def) @@ -1236,6 +1274,7 @@ module TypeInfer = | None -> emptySubst let s = compose sRet (compose s2 s1) applySubstToCapturedLocals telemetry startIndex s + applySubstToCapturedExpressions telemetry expressionStartIndex s let env' = applyEnv s env let inferred = applyType s t1 validateMapKeyTypes span inferred @@ -1253,6 +1292,7 @@ module TypeInfer = | None -> emptySubst let s = compose sRet s1 applySubstToCapturedLocals telemetry startIndex s + applySubstToCapturedExpressions telemetry expressionStartIndex s let env' = applyEnv s env let inferred = applyType s t1 validateMapKeyTypes span inferred @@ -1270,6 +1310,7 @@ module TypeInfer = let s2 = unify typeDefs (applyType s1 tValue) tPat span let s = compose s2 s1 applySubstToCapturedLocals telemetry startIndex s + applySubstToCapturedExpressions telemetry expressionStartIndex s let envBase = applyEnv s env let typedBindings = envPat |> Map.map (fun _ ty -> applyType s ty) typedBindings @@ -1321,6 +1362,7 @@ module TypeInfer = | None -> () applySubstToCapturedLocals telemetry startIndex sRec + applySubstToCapturedExpressions telemetry expressionStartIndex sRec let envGeneralize = applyEnv sRec env let schemes = @@ -1353,16 +1395,22 @@ module TypeInfer = emptySubst let s = compose sDiscard s1 applySubstToCapturedLocals telemetry startIndex s + applySubstToCapturedExpressions telemetry expressionStartIndex s validateMapKeyTypes (Ast.spanOfExpr expr) (applyType s t1) env <- applyEnv s env typed.Add(TSExpr typedExpr) - typed |> Seq.toList, capturedLocalVariableTypes telemetry + let expressionTypes = + match telemetry with + | Some value -> value.ExpressionTypes |> Seq.toList + | None -> [] + typed |> Seq.toList, capturedLocalVariableTypes telemetry, expressionTypes with _ -> reraise () let inferProgramWithExternsRaw (externs: ExternalFunction list) (program: Program) : TypedProgram = - inferProgramWithExternsRawAndLocals false externs program |> fst + let typed, _, _ = inferProgramWithExternsRawAndLocals false externs program + typed let inferProgramWithExterns (externs: ExternalFunction list) (program: Program) : TypedProgram = let reserved = BuiltinSignatures.builtinReservedNames @@ -1392,7 +1440,17 @@ module TypeInfer = |> Option.iter (fun name -> raise (TypeException { Message = $"Top-level binding '{name}' collides with reserved stdlib symbol"; Span = unknownSpan })) - inferProgramWithExternsRawAndLocals true externs program + let typed, locals, _ = inferProgramWithExternsRawAndLocals true externs program + typed, locals + + let inferResolvedProgramWithExterns (externs: ExternalFunction list) (program: Program) : ResolvedProgram = + let typed, _, expressions = inferProgramWithExternsRawAndLocals true externs program + { TypedProgram = typed + ExpressionTypes = expressions + SymbolReferences = [] } + + let inferResolvedProgram (program: Program) : ResolvedProgram = + inferResolvedProgramWithExterns [] program let inferProgramWithLocalVariableTypes (program: Program) : TypedProgram * LocalVariableTypeInfo list = inferProgramWithExternsAndLocalVariableTypes [] program diff --git a/tests/FScript.Language.Tests/FScript.Language.Tests.fsproj b/tests/FScript.Language.Tests/FScript.Language.Tests.fsproj index eb802ed..c24b288 100644 --- a/tests/FScript.Language.Tests/FScript.Language.Tests.fsproj +++ b/tests/FScript.Language.Tests/FScript.Language.Tests.fsproj @@ -13,6 +13,7 @@ + diff --git a/tests/FScript.Language.Tests/ResolvedInferenceTests.fs b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs new file mode 100644 index 0000000..2c55ae8 --- /dev/null +++ b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs @@ -0,0 +1,31 @@ +namespace FScript.Language.Tests + +open FScript.Language +open NUnit.Framework + +[] +type ResolvedInferenceTests() = + [] + member _.``Resolved inference captures every expression without changing legacy output`` () = + let program = Helpers.parse "let add x y = x + y\nadd 1 2" + let legacy = TypeInfer.inferProgram program + let resolved = TypeInfer.inferResolvedProgram program + + Assert.That(resolved.TypedProgram.Length, Is.EqualTo(legacy.Length)) + Assert.That(resolved.ExpressionTypes, Is.Not.Empty) + + let finalApplications = + resolved.ExpressionTypes + |> List.choose (fun item -> + match item.Expr with + | EApply _ -> Some item.Type + | _ -> None) + + Assert.That(finalApplications, Does.Contain(TInt)) + + [] + member _.``Ordinary inference does not allocate resolved telemetry`` () = + let program = Helpers.parse "let identity value = value\nidentity 42" + let resolved = TypeInfer.inferResolvedProgram program + + Assert.That(resolved.ExpressionTypes |> List.exists (fun item -> item.Type = TInt), Is.True) From 417ad50e4e83e94be20b2f69d4c6ca58ac1cce4e Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 16:26:18 +0200 Subject: [PATCH 003/133] feat(compiler): introduce validated typed HIR --- CHANGELOG.md | 1 + FScript.sln | 30 ++++ src/FScript.Compiler/Compiler.fs | 26 +++ src/FScript.Compiler/Diagnostics.fs | 23 +++ src/FScript.Compiler/FScript.Compiler.fsproj | 16 ++ src/FScript.Compiler/Hir.fs | 105 ++++++++++++ src/FScript.Compiler/HirLowering.fs | 154 ++++++++++++++++++ src/FScript.Compiler/HirValidation.fs | 34 ++++ .../FScript.Compiler.Tests.fsproj | 19 +++ tests/FScript.Compiler.Tests/HirTests.fs | 27 +++ tests/FScript.Compiler.Tests/Program.fs | 4 + 11 files changed, 439 insertions(+) create mode 100644 src/FScript.Compiler/Compiler.fs create mode 100644 src/FScript.Compiler/Diagnostics.fs create mode 100644 src/FScript.Compiler/FScript.Compiler.fsproj create mode 100644 src/FScript.Compiler/Hir.fs create mode 100644 src/FScript.Compiler/HirLowering.fs create mode 100644 src/FScript.Compiler/HirValidation.fs create mode 100644 tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj create mode 100644 tests/FScript.Compiler.Tests/HirTests.fs create mode 100644 tests/FScript.Compiler.Tests/Program.fs diff --git a/CHANGELOG.md b/CHANGELOG.md index 8f61934..254ad97 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added a reusable compiler assembly with deterministic typed HIR lowering, validation, diagnostics, and debug output. - Added a compatibility-preserving resolved inference result with expression-level type information for compiler frontends. - Documented the semantic baseline and architecture for typed IR, executable LIR, and compiled backends. - Rewrote the website around concrete FScript behavior and removed vague product slogans. diff --git a/FScript.sln b/FScript.sln index 53e6684..ba48c36 100644 --- a/FScript.sln +++ b/FScript.sln @@ -7,12 +7,16 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "src", "src", "{827E0CD3-B72 EndProject Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "FScript.Language", "src\FScript.Language\FScript.Language.fsproj", "{73C9C013-4E93-45EC-A69D-E490788D8E2B}" EndProject +Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "FScript.Compiler", "src\FScript.Compiler\FScript.Compiler.fsproj", "{D2D66D0D-2D54-4B13-A847-C94732038E17}" +EndProject Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "FScript", "src\FScript\FScript.fsproj", "{56151E59-268A-4A4B-BE94-B7AD2E8B32FA}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "tests", "tests", "{0AB3BF05-4346-4AA6-1389-037BE0695223}" EndProject Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "FScript.Language.Tests", "tests\FScript.Language.Tests\FScript.Language.Tests.fsproj", "{9C62883E-EFB0-4D9E-84F3-4138C123F55E}" EndProject +Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "FScript.Compiler.Tests", "tests\FScript.Compiler.Tests\FScript.Compiler.Tests.fsproj", "{61460CC8-4B1A-4A32-B95B-85DF17DCC54B}" +EndProject Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "FScript.Runtime", "src\FScript.Runtime\FScript.Runtime.fsproj", "{8C2C5767-857A-44B0-80C2-DC90E0A60F4D}" EndProject Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "FScript.Runtime.Tests", "tests\FScript.Runtime.Tests\FScript.Runtime.Tests.fsproj", "{1E2C7B34-04B8-42C9-880D-CC47DEC156A7}" @@ -53,6 +57,18 @@ Global {73C9C013-4E93-45EC-A69D-E490788D8E2B}.Release|x64.Build.0 = Release|Any CPU {73C9C013-4E93-45EC-A69D-E490788D8E2B}.Release|x86.ActiveCfg = Release|Any CPU {73C9C013-4E93-45EC-A69D-E490788D8E2B}.Release|x86.Build.0 = Release|Any CPU + {D2D66D0D-2D54-4B13-A847-C94732038E17}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {D2D66D0D-2D54-4B13-A847-C94732038E17}.Debug|Any CPU.Build.0 = Debug|Any CPU + {D2D66D0D-2D54-4B13-A847-C94732038E17}.Debug|x64.ActiveCfg = Debug|Any CPU + {D2D66D0D-2D54-4B13-A847-C94732038E17}.Debug|x64.Build.0 = Debug|Any CPU + {D2D66D0D-2D54-4B13-A847-C94732038E17}.Debug|x86.ActiveCfg = Debug|Any CPU + {D2D66D0D-2D54-4B13-A847-C94732038E17}.Debug|x86.Build.0 = Debug|Any CPU + {D2D66D0D-2D54-4B13-A847-C94732038E17}.Release|Any CPU.ActiveCfg = Release|Any CPU + {D2D66D0D-2D54-4B13-A847-C94732038E17}.Release|Any CPU.Build.0 = Release|Any CPU + {D2D66D0D-2D54-4B13-A847-C94732038E17}.Release|x64.ActiveCfg = Release|Any CPU + {D2D66D0D-2D54-4B13-A847-C94732038E17}.Release|x64.Build.0 = Release|Any CPU + {D2D66D0D-2D54-4B13-A847-C94732038E17}.Release|x86.ActiveCfg = Release|Any CPU + {D2D66D0D-2D54-4B13-A847-C94732038E17}.Release|x86.Build.0 = Release|Any CPU {56151E59-268A-4A4B-BE94-B7AD2E8B32FA}.Debug|Any CPU.ActiveCfg = Debug|Any CPU {56151E59-268A-4A4B-BE94-B7AD2E8B32FA}.Debug|Any CPU.Build.0 = Debug|Any CPU {56151E59-268A-4A4B-BE94-B7AD2E8B32FA}.Debug|x64.ActiveCfg = Debug|Any CPU @@ -77,6 +93,18 @@ Global {9C62883E-EFB0-4D9E-84F3-4138C123F55E}.Release|x64.Build.0 = Release|Any CPU {9C62883E-EFB0-4D9E-84F3-4138C123F55E}.Release|x86.ActiveCfg = Release|Any CPU {9C62883E-EFB0-4D9E-84F3-4138C123F55E}.Release|x86.Build.0 = Release|Any CPU + {61460CC8-4B1A-4A32-B95B-85DF17DCC54B}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {61460CC8-4B1A-4A32-B95B-85DF17DCC54B}.Debug|Any CPU.Build.0 = Debug|Any CPU + {61460CC8-4B1A-4A32-B95B-85DF17DCC54B}.Debug|x64.ActiveCfg = Debug|Any CPU + {61460CC8-4B1A-4A32-B95B-85DF17DCC54B}.Debug|x64.Build.0 = Debug|Any CPU + {61460CC8-4B1A-4A32-B95B-85DF17DCC54B}.Debug|x86.ActiveCfg = Debug|Any CPU + {61460CC8-4B1A-4A32-B95B-85DF17DCC54B}.Debug|x86.Build.0 = Debug|Any CPU + {61460CC8-4B1A-4A32-B95B-85DF17DCC54B}.Release|Any CPU.ActiveCfg = Release|Any CPU + {61460CC8-4B1A-4A32-B95B-85DF17DCC54B}.Release|Any CPU.Build.0 = Release|Any CPU + {61460CC8-4B1A-4A32-B95B-85DF17DCC54B}.Release|x64.ActiveCfg = Release|Any CPU + {61460CC8-4B1A-4A32-B95B-85DF17DCC54B}.Release|x64.Build.0 = Release|Any CPU + {61460CC8-4B1A-4A32-B95B-85DF17DCC54B}.Release|x86.ActiveCfg = Release|Any CPU + {61460CC8-4B1A-4A32-B95B-85DF17DCC54B}.Release|x86.Build.0 = Release|Any CPU {8C2C5767-857A-44B0-80C2-DC90E0A60F4D}.Debug|Any CPU.ActiveCfg = Debug|Any CPU {8C2C5767-857A-44B0-80C2-DC90E0A60F4D}.Debug|Any CPU.Build.0 = Debug|Any CPU {8C2C5767-857A-44B0-80C2-DC90E0A60F4D}.Debug|x64.ActiveCfg = Debug|Any CPU @@ -191,8 +219,10 @@ Global EndGlobalSection GlobalSection(NestedProjects) = preSolution {73C9C013-4E93-45EC-A69D-E490788D8E2B} = {827E0CD3-B72D-47B6-A68D-7590B98EB39B} + {D2D66D0D-2D54-4B13-A847-C94732038E17} = {827E0CD3-B72D-47B6-A68D-7590B98EB39B} {56151E59-268A-4A4B-BE94-B7AD2E8B32FA} = {827E0CD3-B72D-47B6-A68D-7590B98EB39B} {9C62883E-EFB0-4D9E-84F3-4138C123F55E} = {0AB3BF05-4346-4AA6-1389-037BE0695223} + {61460CC8-4B1A-4A32-B95B-85DF17DCC54B} = {0AB3BF05-4346-4AA6-1389-037BE0695223} {8C2C5767-857A-44B0-80C2-DC90E0A60F4D} = {827E0CD3-B72D-47B6-A68D-7590B98EB39B} {1E2C7B34-04B8-42C9-880D-CC47DEC156A7} = {0AB3BF05-4346-4AA6-1389-037BE0695223} {B734E1E1-59C2-47E0-8D19-A9C5C95938F1} = {0AB3BF05-4346-4AA6-1389-037BE0695223} diff --git a/src/FScript.Compiler/Compiler.fs b/src/FScript.Compiler/Compiler.fs new file mode 100644 index 0000000..c6b8134 --- /dev/null +++ b/src/FScript.Compiler/Compiler.fs @@ -0,0 +1,26 @@ +namespace FScript.Compiler + +open FScript.Language + +type CompileInput = + { Program: Program + Externs: ExternalFunction list } + +type CompiledProgram = + { Hir: Hir.Program } + +module Compiler = + let compile input = + try + input.Program + |> TypeInfer.inferResolvedProgramWithExterns input.Externs + |> HirLowering.lower + |> Result.bind HirValidation.validate + |> Result.map (fun hir -> { Hir = hir }) + with + | TypeException error -> + Error + [ { Code = "FSH1000" + Stage = Hir + Message = error.Message + Span = Some error.Span } ] diff --git a/src/FScript.Compiler/Diagnostics.fs b/src/FScript.Compiler/Diagnostics.fs new file mode 100644 index 0000000..e6f612a --- /dev/null +++ b/src/FScript.Compiler/Diagnostics.fs @@ -0,0 +1,23 @@ +namespace FScript.Compiler + +open FScript.Language + +type DiagnosticStage = + | Hir + | Lowering + | Interpreter + | GoBackend + | Artifact + +type CompilerDiagnostic = + { Code: string + Stage: DiagnosticStage + Message: string + Span: Span option } + +module Diagnostic = + let hir code message span = + { Code = code + Stage = Hir + Message = message + Span = span } diff --git a/src/FScript.Compiler/FScript.Compiler.fsproj b/src/FScript.Compiler/FScript.Compiler.fsproj new file mode 100644 index 0000000..23fa77a --- /dev/null +++ b/src/FScript.Compiler/FScript.Compiler.fsproj @@ -0,0 +1,16 @@ + + + net10.0 + false + + + + + + + + + + + + diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs new file mode 100644 index 0000000..f43a82b --- /dev/null +++ b/src/FScript.Compiler/Hir.fs @@ -0,0 +1,105 @@ +namespace FScript.Compiler + +open FScript.Language + +module Hir = + type SymbolId = SymbolId of int + + type LiteralValue = + | Unit + | Int of int64 + | Float of float + | Bool of bool + | String of string + + type PrimitiveOperation = + | IntAdd + | FloatAdd + | IntSubtract + | FloatSubtract + | IntMultiply + | FloatMultiply + | IntDivide + | FloatDivide + | IntRemainder + | StructuralEqual + | NumericLessThan + | NumericGreaterThan + | NumericLessThanOrEqual + | NumericGreaterThanOrEqual + | BooleanAnd + | BooleanOr + + type Expression = + { Type: Type + Span: Span + Node: ExpressionNode } + + and ExpressionNode = + | Constant of LiteralValue + | Reference of SymbolId + | Lambda of SymbolId * string * Type * Expression + | Apply of Expression * Expression + | If of Expression * Expression * Expression + | Primitive of PrimitiveOperation * Expression list + + type Declaration = + { Id: SymbolId + Name: string + Scheme: Scheme + IsRecursive: bool + IsExported: bool + Span: Span + Value: Expression } + + type Program = + { Declarations: Declaration list + Entry: Expression option } + + let symbolNumber (SymbolId value) = value + + let rec formatType = Types.typeToString + + let private formatLiteral value = + match value with + | Unit -> "unit" + | Int value -> $"int {value}" + | Float value -> $"float {value:R}" + | Bool value -> $"bool {value.ToString().ToLowerInvariant()}" + | String value -> $"string {System.Text.Json.JsonSerializer.Serialize(value)}" + + let private formatPrimitive value = sprintf "%A" value + + let format (program: Program) = + let lines = ResizeArray() + let rec expression indent value = + let prefix = System.String(' ', indent) + let typed text = $"{prefix}{text} : {formatType value.Type}" + match value.Node with + | Constant literal -> lines.Add(typed $"const {formatLiteral literal}") + | Reference id -> lines.Add(typed (sprintf "ref %%%d" (symbolNumber id))) + | Lambda(id, name, parameterType, body) -> + lines.Add(typed (sprintf "lambda %%%d %s: %s" (symbolNumber id) name (formatType parameterType))) + expression (indent + 2) body + | Apply(fn, argument) -> + lines.Add(typed "apply") + expression (indent + 2) fn + expression (indent + 2) argument + | If(condition, whenTrue, whenFalse) -> + lines.Add(typed "if") + expression (indent + 2) condition + expression (indent + 2) whenTrue + expression (indent + 2) whenFalse + | Primitive(operation, arguments) -> + lines.Add(typed $"primitive {formatPrimitive operation}") + arguments |> List.iter (expression (indent + 2)) + + for declaration in program.Declarations do + lines.Add(sprintf "decl %%%d %s: %s" (symbolNumber declaration.Id) declaration.Name (formatType (let (Forall(_, value)) = declaration.Scheme in value))) + expression 2 declaration.Value + match program.Entry with + | Some entry -> + lines.Add("entry") + expression 2 entry + | None -> () + System.String.Join("\n", lines) + "\n" diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs new file mode 100644 index 0000000..d2ad521 --- /dev/null +++ b/src/FScript.Compiler/HirLowering.fs @@ -0,0 +1,154 @@ +namespace FScript.Compiler + +open FScript.Language + +module HirLowering = + type private State = + { mutable NextSymbol: int + ExpressionTypes: Map + mutable Diagnostics: CompilerDiagnostic list } + + let private newSymbol state = + let value = state.NextSymbol + state.NextSymbol <- value + 1 + Hir.SymbolId value + + let private expressionType state expression fallback = + state.ExpressionTypes |> Map.tryFind expression |> Option.defaultValue fallback + + let private literal = function + | LInt value -> Hir.Int value, TInt + | LFloat value -> Hir.Float value, TFloat + | LBool value -> Hir.Bool value, TBool + | LString value -> Hir.String value, TString + + let private operation op operandType = + match op, operandType with + | "+", TInt -> Some Hir.IntAdd + | "+", TFloat -> Some Hir.FloatAdd + | "-", TInt -> Some Hir.IntSubtract + | "-", TFloat -> Some Hir.FloatSubtract + | "*", TInt -> Some Hir.IntMultiply + | "*", TFloat -> Some Hir.FloatMultiply + | "/", TInt -> Some Hir.IntDivide + | "/", TFloat -> Some Hir.FloatDivide + | "%", TInt -> Some Hir.IntRemainder + | "=", _ -> Some Hir.StructuralEqual + | "<", _ -> Some Hir.NumericLessThan + | ">", _ -> Some Hir.NumericGreaterThan + | "<=", _ -> Some Hir.NumericLessThanOrEqual + | ">=", _ -> Some Hir.NumericGreaterThanOrEqual + | "&&", TBool -> Some Hir.BooleanAnd + | "||", TBool -> Some Hir.BooleanOr + | _ -> None + + let rec private lowerExpression state environment fallback expression = + let span = Ast.spanOfExpr expression + let inferred = expressionType state expression fallback + let make node = ({ Type = inferred; Span = span; Node = node }: Hir.Expression) + match expression with + | EUnit _ -> Ok(make (Hir.Constant Hir.Unit)) + | ELiteral(value, _) -> + let value, _ = literal value + Ok(make (Hir.Constant value)) + | EParen(inner, _) -> lowerExpression state environment inferred inner + | EVar(name, _) -> + match environment |> Map.tryFind name with + | Some id -> Ok(make (Hir.Reference id)) + | None -> Error [ Diagnostic.hir "FSH1001" $"Unresolved HIR reference '{name}'." (Some span) ] + | ELambda(parameter, body, _) -> + match inferred with + | TFun(parameterType, resultType) -> + let id = newSymbol state + lowerExpression state (environment |> Map.add parameter.Name id) resultType body + |> Result.map (fun lowered -> make (Hir.Lambda(id, parameter.Name, parameterType, lowered))) + | _ -> Error [ Diagnostic.hir "FSH1002" "Lambda does not have a function type." (Some span) ] + | EApply(fn, argument, _) -> + let fnType = expressionType state fn (TFun(Types.freshVar(), inferred)) + let argumentType = + match fnType with + | TFun(value, _) -> value + | _ -> Types.freshVar() + match lowerExpression state environment fnType fn, lowerExpression state environment argumentType argument with + | Ok loweredFn, Ok loweredArgument -> Ok(make (Hir.Apply(loweredFn, loweredArgument))) + | Error left, Error right -> Error(left @ right) + | Error errors, _ + | _, Error errors -> Error errors + | EIf(condition, whenTrue, whenFalse, _) -> + match lowerExpression state environment TBool condition, + lowerExpression state environment inferred whenTrue, + lowerExpression state environment inferred whenFalse with + | Ok c, Ok t, Ok f -> Ok(make (Hir.If(c, t, f))) + | results -> + let errors = + [ match results with + | Error e, _, _ -> yield! e + | _ -> () + match results with + | _, Error e, _ -> yield! e + | _ -> () + match results with + | _, _, Error e -> yield! e + | _ -> () ] + Error errors + | EBinOp(op, left, right, _) -> + let capturedOperandType = expressionType state left inferred + let operandType = + match capturedOperandType, inferred with + | TVar _, (TInt | TFloat as concrete) -> concrete + | value, _ -> value + match operation op operandType with + | None -> Error [ Diagnostic.hir "FSH1003" $"Unresolved primitive operation '{op}'." (Some span) ] + | Some primitive -> + match lowerExpression state environment operandType left, lowerExpression state environment operandType right with + | Ok l, Ok r -> Ok(make (Hir.Primitive(primitive, [ l; r ]))) + | Error l, Error r -> Error(l @ r) + | Error errors, _ + | _, Error errors -> Error errors + | _ -> Error [ Diagnostic.hir "FSH1099" $"HIR lowering does not yet support {expression.GetType().Name}." (Some span) ] + + let lower (resolved: TypeInfer.ResolvedProgram) : Result = + let expressionTypes = + resolved.ExpressionTypes + |> List.fold (fun state item -> state |> Map.add item.Expr item.Type) Map.empty + let state = { NextSymbol = 0; ExpressionTypes = expressionTypes; Diagnostics = [] } + let builtins = + BuiltinSignatures.builtinSchemes + |> Map.toList + |> List.sortBy fst + |> List.map (fun (name, _) -> name, newSymbol state) + |> Map.ofList + let topLevel = + resolved.TypedProgram + |> List.collect (function + | TypeInfer.TSLet(name, _, _, _, _, _) -> [ name ] + | TypeInfer.TSLetRecGroup(bindings, _, _) -> bindings |> List.map (fun (name, _, _, _) -> name) + | _ -> []) + |> List.distinct + |> List.fold (fun env name -> env |> Map.add name (newSymbol state)) builtins + + let mutable declarations = [] + let mutable entry = None + let mutable errors = [] + for statement in resolved.TypedProgram do + match statement with + | TypeInfer.TSLet(name, expression, inferred, isRecursive, isExported, span) -> + match lowerExpression state topLevel inferred expression with + | Ok value -> + declarations <- declarations @ + [ ({ Id = Map.find name topLevel; + Name = name; + Scheme = Types.generalize Map.empty inferred; + IsRecursive = isRecursive; + IsExported = isExported; + Span = span; + Value = value }: Hir.Declaration) ] + | Error diagnostics -> errors <- errors @ diagnostics + | TypeInfer.TSExpr expression -> + match lowerExpression state topLevel expression.Type expression.Expr with + | Ok value -> entry <- Some value + | Error diagnostics -> errors <- errors @ diagnostics + | TypeInfer.TSType _ -> () + | _ -> errors <- errors @ [ Diagnostic.hir "FSH1098" "Top-level declaration form is not yet supported by HIR lowering." None ] + if errors.IsEmpty then Ok { Declarations = declarations; Entry = entry } + else Error errors diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs new file mode 100644 index 0000000..d06b4f4 --- /dev/null +++ b/src/FScript.Compiler/HirValidation.fs @@ -0,0 +1,34 @@ +namespace FScript.Compiler + +module HirValidation = + let validate (program: Hir.Program) = + let declarationIds = program.Declarations |> List.map (fun value -> value.Id) + let duplicateIds = declarationIds |> List.countBy id |> List.filter (fun (_, count) -> count > 1) + if not duplicateIds.IsEmpty then + Error [ Diagnostic.hir "FSH1101" "HIR declaration IDs must be unique." None ] + else + let rec references (expression: Hir.Expression) = + match expression.Node with + | Hir.Constant _ -> Set.empty + | Hir.Reference id -> Set.singleton id + | Hir.Lambda(parameter, _, _, body) -> references body |> Set.remove parameter + | Hir.Apply(fn, argument) -> Set.union (references fn) (references argument) + | Hir.If(condition, whenTrue, whenFalse) -> + Set.unionMany [ references condition; references whenTrue; references whenFalse ] + | Hir.Primitive(_, arguments) -> arguments |> List.map references |> Set.unionMany + let declared = declarationIds |> Set.ofList + let unresolved = + [ yield! program.Declarations |> List.collect (fun value -> references value.Value |> Set.toList) + match program.Entry with + | Some value -> yield! references value |> Set.toList + | None -> () ] + |> Set.ofList + |> Set.filter (fun id -> not (Set.contains id declared)) + // IDs below the first declaration are reserved deterministic builtin symbols. + let firstDeclaration = + match declarationIds |> List.map Hir.symbolNumber with + | [] -> 0 + | values -> List.min values + let invalid = unresolved |> Set.filter (fun id -> Hir.symbolNumber id >= firstDeclaration) + if invalid.IsEmpty then Ok program + else Error [ Diagnostic.hir "FSH1102" "HIR contains unresolved symbol references." None ] diff --git a/tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj b/tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj new file mode 100644 index 0000000..3c8f566 --- /dev/null +++ b/tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj @@ -0,0 +1,19 @@ + + + net10.0 + false + false + + + + + + + + + + + + + + diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs new file mode 100644 index 0000000..1688da7 --- /dev/null +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -0,0 +1,27 @@ +namespace FScript.Compiler.Tests + +open FScript.Compiler +open FScript.Language +open NUnit.Framework + +[] +type HirTests() = + [] + member _.``Lowers deterministic typed add program`` () = + let source = "let add x y = x + y\nadd 1 2" + let input = { Program = Parser.parseProgram source; Externs = [] } + match Compiler.compile input with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + let actual = Hir.format compiled.Hir + Assert.That(actual, Does.Contain("decl %")) + Assert.That(actual, Does.Contain("primitive IntAdd")) + Assert.That(actual, Does.Contain("entry")) + Assert.That(Hir.format compiled.Hir, Is.EqualTo(actual)) + + [] + member _.``Reports unsupported HIR forms as diagnostics`` () = + let input = { Program = Parser.parseProgram "[1; 2]"; Externs = [] } + match Compiler.compile input with + | Ok _ -> Assert.Fail("Expected unsupported-form diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSH1099")) diff --git a/tests/FScript.Compiler.Tests/Program.fs b/tests/FScript.Compiler.Tests/Program.fs new file mode 100644 index 0000000..b2d56f2 --- /dev/null +++ b/tests/FScript.Compiler.Tests/Program.fs @@ -0,0 +1,4 @@ +module FScript.Compiler.Tests.Program + +[] +let main _ = 0 From b5ba5b489ca0eea9ac8d7688bc250b6bb34c0b5c Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 16:28:18 +0200 Subject: [PATCH 004/133] feat(compiler): execute core FScript LIR --- CHANGELOG.md | 1 + src/FScript.Compiler/Compiler.fs | 7 +- src/FScript.Compiler/FScript.Compiler.fsproj | 2 + src/FScript.Compiler/Lir.fs | 93 +++++++++++++++++++ src/FScript.Compiler/LirInterpreter.fs | 98 ++++++++++++++++++++ tests/FScript.Compiler.Tests/HirTests.fs | 3 + 6 files changed, 202 insertions(+), 2 deletions(-) create mode 100644 src/FScript.Compiler/Lir.fs create mode 100644 src/FScript.Compiler/LirInterpreter.fs diff --git a/CHANGELOG.md b/CHANGELOG.md index 254ad97..376e0c1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added executable LIR lowering and interpretation for primitive, curried-function, application, and conditional programs. - Added a reusable compiler assembly with deterministic typed HIR lowering, validation, diagnostics, and debug output. - Added a compatibility-preserving resolved inference result with expression-level type information for compiler frontends. - Documented the semantic baseline and architecture for typed IR, executable LIR, and compiled backends. diff --git a/src/FScript.Compiler/Compiler.fs b/src/FScript.Compiler/Compiler.fs index c6b8134..55e1497 100644 --- a/src/FScript.Compiler/Compiler.fs +++ b/src/FScript.Compiler/Compiler.fs @@ -7,7 +7,8 @@ type CompileInput = Externs: ExternalFunction list } type CompiledProgram = - { Hir: Hir.Program } + { Hir: Hir.Program + Lir: Lir.Program } module Compiler = let compile input = @@ -16,7 +17,9 @@ module Compiler = |> TypeInfer.inferResolvedProgramWithExterns input.Externs |> HirLowering.lower |> Result.bind HirValidation.validate - |> Result.map (fun hir -> { Hir = hir }) + |> Result.bind (fun hir -> + hir |> LirLowering.lower |> LirValidation.validate + |> Result.map (fun lir -> { Hir = hir; Lir = lir })) with | TypeException error -> Error diff --git a/src/FScript.Compiler/FScript.Compiler.fsproj b/src/FScript.Compiler/FScript.Compiler.fsproj index 23fa77a..f4ea8d8 100644 --- a/src/FScript.Compiler/FScript.Compiler.fsproj +++ b/src/FScript.Compiler/FScript.Compiler.fsproj @@ -8,6 +8,8 @@ + + diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs new file mode 100644 index 0000000..458913d --- /dev/null +++ b/src/FScript.Compiler/Lir.fs @@ -0,0 +1,93 @@ +namespace FScript.Compiler + +open FScript.Language + +module Lir = + type SymbolId = SymbolId of int + + type Constant = + | Unit + | Int of int64 + | Float of float + | Bool of bool + | String of string + + type Operation = + | IntAdd | FloatAdd | IntSubtract | FloatSubtract + | IntMultiply | FloatMultiply | IntDivide | FloatDivide | IntRemainder + | StructuralEqual + | NumericLessThan | NumericGreaterThan | NumericLessThanOrEqual | NumericGreaterThanOrEqual + | BooleanAnd | BooleanOr + + type Expression = + { Type: Type + Span: Span + Node: ExpressionNode } + and ExpressionNode = + | Constant of Constant + | Load of SymbolId + | MakeClosure of SymbolId * string * Type * Expression + | CallClosure of Expression * Expression + | Branch of Expression * Expression * Expression + | Operation of Operation * Expression list + + type Declaration = + { Id: SymbolId + Name: string + Type: Type + IsRecursive: bool + IsExported: bool + Span: Span + Value: Expression } + + type Program = + { Declarations: Declaration list + Entry: Expression option } + + let symbolNumber (SymbolId value) = value + +module LirLowering = + let private symbol (Hir.SymbolId value) = Lir.SymbolId value + + let private operation = function + | Hir.IntAdd -> Lir.IntAdd | Hir.FloatAdd -> Lir.FloatAdd + | Hir.IntSubtract -> Lir.IntSubtract | Hir.FloatSubtract -> Lir.FloatSubtract + | Hir.IntMultiply -> Lir.IntMultiply | Hir.FloatMultiply -> Lir.FloatMultiply + | Hir.IntDivide -> Lir.IntDivide | Hir.FloatDivide -> Lir.FloatDivide + | Hir.IntRemainder -> Lir.IntRemainder | Hir.StructuralEqual -> Lir.StructuralEqual + | Hir.NumericLessThan -> Lir.NumericLessThan | Hir.NumericGreaterThan -> Lir.NumericGreaterThan + | Hir.NumericLessThanOrEqual -> Lir.NumericLessThanOrEqual + | Hir.NumericGreaterThanOrEqual -> Lir.NumericGreaterThanOrEqual + | Hir.BooleanAnd -> Lir.BooleanAnd | Hir.BooleanOr -> Lir.BooleanOr + + let private constant = function + | Hir.Unit -> Lir.Unit | Hir.Int value -> Lir.Int value | Hir.Float value -> Lir.Float value + | Hir.Bool value -> Lir.Bool value | Hir.String value -> Lir.String value + + let rec private expression (value: Hir.Expression) : Lir.Expression = + let node = + match value.Node with + | Hir.Constant value -> Lir.Constant(constant value) + | Hir.Reference id -> Lir.Load(symbol id) + | Hir.Lambda(id, name, parameterType, body) -> Lir.MakeClosure(symbol id, name, parameterType, expression body) + | Hir.Apply(fn, argument) -> Lir.CallClosure(expression fn, expression argument) + | Hir.If(condition, whenTrue, whenFalse) -> Lir.Branch(expression condition, expression whenTrue, expression whenFalse) + | Hir.Primitive(op, arguments) -> Lir.Operation(operation op, arguments |> List.map expression) + { Type = value.Type; Span = value.Span; Node = node } + + let lower (program: Hir.Program) : Lir.Program = + { Declarations = + program.Declarations + |> List.map (fun value -> + let (Forall(_, declarationType)) = value.Scheme + ({ Id = symbol value.Id; Name = value.Name; Type = declarationType + IsRecursive = value.IsRecursive; IsExported = value.IsExported + Span = value.Span; Value = expression value.Value }: Lir.Declaration)) + Entry = program.Entry |> Option.map expression } + +module LirValidation = + let validate (program: Lir.Program) = + let ids = program.Declarations |> List.map _.Id + if ids.Length <> (ids |> Set.ofList |> Set.count) then + Error [ { Code = "FSL2001"; Stage = Lowering; Message = "LIR declaration IDs must be unique."; Span = None } ] + else Ok program diff --git a/src/FScript.Compiler/LirInterpreter.fs b/src/FScript.Compiler/LirInterpreter.fs new file mode 100644 index 0000000..2d5bd61 --- /dev/null +++ b/src/FScript.Compiler/LirInterpreter.fs @@ -0,0 +1,98 @@ +namespace FScript.Compiler + +open FScript.Language + +type InterpreterContext = + { ScriptName: string option + Arguments: string list } + +type LirValue = + | VUnit + | VInt of int64 + | VFloat of float + | VBool of bool + | VString of string + | VClosure of Lir.SymbolId * string * Lir.Expression * Map ref + +type ExecutionResult = { Value: LirValue } + +module LirInterpreter = + let rec private equal left right = + match left, right with + | VUnit, VUnit -> true + | VInt a, VInt b -> a = b + | VFloat a, VFloat b -> a = b + | VBool a, VBool b -> a = b + | VString a, VString b -> a = b + | _ -> false + + let private fail code span message = + Error [ { Code = code; Stage = Interpreter; Message = message; Span = Some span } ] + + let rec private evaluate environment (expression: Lir.Expression) = + let binary work = + match expression.Node with + | Lir.Operation(_, [ left; right ]) -> + match evaluate environment left, evaluate environment right with + | Ok a, Ok b -> work a b + | Error a, Error b -> Error(a @ b) + | Error errors, _ | _, Error errors -> Error errors + | _ -> fail "FSI3002" expression.Span "Invalid primitive arity." + match expression.Node with + | Lir.Constant value -> + Ok(match value with Lir.Unit -> VUnit | Lir.Int v -> VInt v | Lir.Float v -> VFloat v | Lir.Bool v -> VBool v | Lir.String v -> VString v) + | Lir.Load id -> + environment |> Map.tryFind id |> Option.map Ok + |> Option.defaultWith (fun () -> fail "FSI3001" expression.Span (sprintf "Unknown LIR symbol %%%d." (Lir.symbolNumber id))) + | Lir.MakeClosure(id, name, _, body) -> Ok(VClosure(id, name, body, ref environment)) + | Lir.CallClosure(fn, argument) -> + match evaluate environment fn, evaluate environment argument with + | Ok(VClosure(parameter, _, body, captured)), Ok value -> evaluate (captured.Value |> Map.add parameter value) body + | Ok _, Ok _ -> fail "FSI3003" expression.Span "Attempted to call a non-function value." + | Error a, Error b -> Error(a @ b) + | Error errors, _ | _, Error errors -> Error errors + | Lir.Branch(condition, whenTrue, whenFalse) -> + match evaluate environment condition with + | Ok(VBool true) -> evaluate environment whenTrue + | Ok(VBool false) -> evaluate environment whenFalse + | Ok _ -> fail "FSI3004" expression.Span "Branch condition is not Boolean." + | Error errors -> Error errors + | Lir.Operation(op, _) -> + binary (fun left right -> + match op, left, right with + | Lir.IntAdd, VInt a, VInt b -> Ok(VInt(a + b)) + | Lir.FloatAdd, VFloat a, VFloat b -> Ok(VFloat(a + b)) + | Lir.IntSubtract, VInt a, VInt b -> Ok(VInt(a - b)) + | Lir.FloatSubtract, VFloat a, VFloat b -> Ok(VFloat(a - b)) + | Lir.IntMultiply, VInt a, VInt b -> Ok(VInt(a * b)) + | Lir.FloatMultiply, VFloat a, VFloat b -> Ok(VFloat(a * b)) + | Lir.IntDivide, VInt _, VInt 0L -> fail "FSI3005" expression.Span "Division by zero." + | Lir.IntDivide, VInt a, VInt b -> Ok(VInt(a / b)) + | Lir.FloatDivide, VFloat a, VFloat b -> Ok(VFloat(a / b)) + | Lir.IntRemainder, VInt _, VInt 0L -> fail "FSI3005" expression.Span "Division by zero." + | Lir.IntRemainder, VInt a, VInt b -> Ok(VInt(a % b)) + | Lir.StructuralEqual, a, b -> Ok(VBool(equal a b)) + | Lir.NumericLessThan, VInt a, VInt b -> Ok(VBool(a < b)) + | Lir.NumericGreaterThan, VInt a, VInt b -> Ok(VBool(a > b)) + | Lir.NumericLessThanOrEqual, VInt a, VInt b -> Ok(VBool(a <= b)) + | Lir.NumericGreaterThanOrEqual, VInt a, VInt b -> Ok(VBool(a >= b)) + | Lir.BooleanAnd, VBool a, VBool b -> Ok(VBool(a && b)) + | Lir.BooleanOr, VBool a, VBool b -> Ok(VBool(a || b)) + | _ -> fail "FSI3006" expression.Span "Primitive operands do not match their LIR operation.") + + let execute (_context: InterpreterContext) (program: Lir.Program) = + let environment = ref Map.empty + let mutable failure = None + for declaration in program.Declarations do + if failure.IsNone then + match evaluate environment.Value declaration.Value with + | Error errors -> failure <- Some errors + | Ok(VClosure(id, name, body, captured) as closure) -> + let next = environment.Value |> Map.add declaration.Id closure + if declaration.IsRecursive then captured.Value <- next + environment.Value <- next + | Ok value -> environment.Value <- environment.Value |> Map.add declaration.Id value + match failure, program.Entry with + | Some errors, _ -> Error errors + | None, Some entry -> evaluate environment.Value entry |> Result.map (fun value -> { Value = value }) + | None, None -> Ok { Value = VUnit } diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index 1688da7..a47443d 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -18,6 +18,9 @@ type HirTests() = Assert.That(actual, Does.Contain("primitive IntAdd")) Assert.That(actual, Does.Contain("entry")) Assert.That(Hir.format compiled.Hir, Is.EqualTo(actual)) + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 3L)) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) [] member _.``Reports unsupported HIR forms as diagnostics`` () = From 0a9bd52c7728def2e6b713fc09d5fe125007eb93 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 16:30:58 +0200 Subject: [PATCH 005/133] feat(go): generate minimal FScript Go projects --- CHANGELOG.md | 1 + src/FScript.Compiler/FScript.Compiler.fsproj | 1 + src/FScript.Compiler/GoBackend.fs | 106 ++++++++++++++++++ .../FScript.Compiler.Tests.fsproj | 1 + .../FScript.Compiler.Tests/GoBackendTests.fs | 47 ++++++++ 5 files changed, 156 insertions(+) create mode 100644 src/FScript.Compiler/GoBackend.fs create mode 100644 tests/FScript.Compiler.Tests/GoBackendTests.fs diff --git a/CHANGELOG.md b/CHANGELOG.md index 376e0c1..1cf9806 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added deterministic Go 1.25 project generation for the initial primitive and curried-function compiler slice. - Added executable LIR lowering and interpretation for primitive, curried-function, application, and conditional programs. - Added a reusable compiler assembly with deterministic typed HIR lowering, validation, diagnostics, and debug output. - Added a compatibility-preserving resolved inference result with expression-level type information for compiler frontends. diff --git a/src/FScript.Compiler/FScript.Compiler.fsproj b/src/FScript.Compiler/FScript.Compiler.fsproj index f4ea8d8..630a81a 100644 --- a/src/FScript.Compiler/FScript.Compiler.fsproj +++ b/src/FScript.Compiler/FScript.Compiler.fsproj @@ -10,6 +10,7 @@ + diff --git a/src/FScript.Compiler/GoBackend.fs b/src/FScript.Compiler/GoBackend.fs new file mode 100644 index 0000000..8d8b6de --- /dev/null +++ b/src/FScript.Compiler/GoBackend.fs @@ -0,0 +1,106 @@ +namespace FScript.Compiler + +open System +open System.Text +open FScript.Language + +type GoOptions = { ModuleName: string } +type GeneratedFile = { RelativePath: string; Content: string } + +module GoBackend = + let rec private goType = function + | TUnit -> "struct{}" + | TInt -> "int64" + | TFloat -> "float64" + | TBool -> "bool" + | TString -> "string" + | TVar _ -> "any" + | TFun(argument, result) -> $"func({goType argument}) {goType result}" + | value -> invalidArg "value" $"Unsupported Go representation for {Types.typeToString value}." + + let private identifier prefix id = $"{prefix}{Lir.symbolNumber id}" + + let private constant = function + | Lir.Unit -> "struct{}{}" + | Lir.Int value -> $"int64({value})" + | Lir.Float value -> value.ToString("R", Globalization.CultureInfo.InvariantCulture) + | Lir.Bool true -> "true" + | Lir.Bool false -> "false" + | Lir.String value -> System.Text.Json.JsonSerializer.Serialize(value) + + let private operation = function + | Lir.IntAdd | Lir.FloatAdd -> Some "+" + | Lir.IntSubtract | Lir.FloatSubtract -> Some "-" + | Lir.IntMultiply | Lir.FloatMultiply -> Some "*" + | Lir.IntDivide | Lir.FloatDivide -> Some "/" + | Lir.IntRemainder -> Some "%" + | Lir.StructuralEqual -> Some "==" + | Lir.NumericLessThan -> Some "<" + | Lir.NumericGreaterThan -> Some ">" + | Lir.NumericLessThanOrEqual -> Some "<=" + | Lir.NumericGreaterThanOrEqual -> Some ">=" + | Lir.BooleanAnd -> Some "&&" + | Lir.BooleanOr -> Some "||" + + let rec private emitExpression environment (expression: Lir.Expression) = + match expression.Node with + | Lir.Constant value -> Ok(constant value) + | Lir.Load id -> + environment |> Map.tryFind id + |> Option.map Ok + |> Option.defaultValue (Error(sprintf "Unknown symbol %%%d" (Lir.symbolNumber id))) + | Lir.MakeClosure(parameter, _, parameterType, body) -> + let name = identifier "arg" parameter + emitExpression (environment |> Map.add parameter name) body + |> Result.map (fun bodyCode -> $"func({name} {goType parameterType}) {goType body.Type} {{ return {bodyCode} }}") + | Lir.CallClosure(fn, argument) -> + match emitExpression environment fn, emitExpression environment argument with + | Ok fnCode, Ok argumentCode -> Ok $"({fnCode})({argumentCode})" + | Error error, _ | _, Error error -> Error error + | Lir.Branch(condition, whenTrue, whenFalse) -> + match emitExpression environment condition, emitExpression environment whenTrue, emitExpression environment whenFalse with + | Ok c, Ok t, Ok f -> Ok $"func() {goType expression.Type} {{ if {c} {{ return {t} }}; return {f} }}()" + | Error error, _, _ | _, Error error, _ | _, _, Error error -> Error error + | Lir.Operation(op, [ left; right ]) -> + match operation op, emitExpression environment left, emitExpression environment right with + | Some token, Ok l, Ok r -> + let l, r = + match op with + | Lir.IntAdd | Lir.IntSubtract | Lir.IntMultiply | Lir.IntDivide | Lir.IntRemainder -> + $"fsruntime.Int({l})", $"fsruntime.Int({r})" + | _ -> l, r + Ok $"({l} {token} {r})" + | None, _, _ -> Error $"Unsupported Go operation {op}" + | _, Error error, _ | _, _, Error error -> Error error + | Lir.Operation _ -> Error "Invalid operation arity" + + let generate options (program: Lir.Program) : Result = + try + let names = + program.Declarations + |> List.fold (fun env declaration -> env |> Map.add declaration.Id (identifier "value" declaration.Id)) Map.empty + let declarations = StringBuilder() + let mutable error = None + for declaration in program.Declarations do + match emitExpression names declaration.Value with + | Ok code -> declarations.AppendLine($"var {names.[declaration.Id]} = {code}") |> ignore + | Error message -> error <- Some(message, declaration.Span) + let entry = + match program.Entry with + | None -> Ok "struct{}{}" + | Some value -> emitExpression names value + match error, entry with + | Some(message, span), _ -> + Error [ { Code = "FSB4001"; Stage = GoBackend; Message = message; Span = Some span } ] + | None, Error message -> + Error [ { Code = "FSB4001"; Stage = GoBackend; Message = message; Span = None } ] + | None, Ok entryCode -> + let moduleName = if String.IsNullOrWhiteSpace options.ModuleName then "fscript.generated/program" else options.ModuleName + let programSource = $"package fscriptprogram\n\nimport \"{moduleName}/internal/fsruntime\"\n\n{declarations}\nfunc Run() any {{ return {entryCode} }}\n" + Ok + [ { RelativePath = "go.mod"; Content = $"module {moduleName}\n\ngo 1.25\n" } + { RelativePath = "main.go"; Content = $"package main\n\nimport (\n\t\"fmt\"\n\t\"{moduleName}/internal/fscriptprogram\"\n)\n\nfunc main() {{ fmt.Println(fscriptprogram.Run()) }}\n" } + { RelativePath = "internal/fscriptprogram/program.go"; Content = programSource } + { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nfunc Int(value any) int64 { return value.(int64) }\n" } ] + with ex -> + Error [ { Code = "FSB4002"; Stage = GoBackend; Message = ex.Message; Span = None } ] diff --git a/tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj b/tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj index 3c8f566..1c23861 100644 --- a/tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj +++ b/tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj @@ -6,6 +6,7 @@ + diff --git a/tests/FScript.Compiler.Tests/GoBackendTests.fs b/tests/FScript.Compiler.Tests/GoBackendTests.fs new file mode 100644 index 0000000..d893a5d --- /dev/null +++ b/tests/FScript.Compiler.Tests/GoBackendTests.fs @@ -0,0 +1,47 @@ +namespace FScript.Compiler.Tests + +open System +open System.Diagnostics +open System.IO +open FScript.Compiler +open FScript.Language +open NUnit.Framework + +[] +type GoBackendTests() = + [] + member _.``Generated add project runs with Go`` () = + let input = { Program = Parser.parseProgram "let add x y = x + y\nadd 1 2"; Externs = [] } + let compiled = + match Compiler.compile input with + | Ok value -> value + | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + let files = + match GoBackend.generate { ModuleName = "fscript.generated/testprogram" } compiled.Lir with + | Ok value -> value + | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + let directory = Path.Combine(Path.GetTempPath(), $"fscript-go-{Guid.NewGuid():N}") + try + for file in files do + let path = Path.Combine(directory, file.RelativePath) + match Path.GetDirectoryName(path) with + | null -> () + | parent -> Directory.CreateDirectory(parent) |> ignore + File.WriteAllText(path, file.Content) + let start = ProcessStartInfo("go") + start.WorkingDirectory <- directory + start.ArgumentList.Add("run") + start.ArgumentList.Add(".") + start.RedirectStandardOutput <- true + start.RedirectStandardError <- true + use proc = + match Process.Start(start) with + | null -> failwith "Failed to start Go" + | value -> value + let output = proc.StandardOutput.ReadToEnd() + let error = proc.StandardError.ReadToEnd() + proc.WaitForExit() + Assert.That(proc.ExitCode, Is.EqualTo(0), error) + Assert.That(output.Trim(), Is.EqualTo("3")) + finally + if Directory.Exists directory then Directory.Delete(directory, true) From 60d0f64e9cae64ff29b8ee230188e43ed5bfa929 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 16:36:28 +0200 Subject: [PATCH 006/133] feat(compiler): add versioned LIR artifacts --- CHANGELOG.md | 1 + src/FScript.Compiler/FScript.Compiler.fsproj | 1 + src/FScript.Compiler/IrArtifact.fs | 181 ++++++++++++++++++ .../FScript.Compiler.Tests.fsproj | 1 + .../FScript.Compiler.Tests/IrArtifactTests.fs | 41 ++++ 5 files changed, 225 insertions(+) create mode 100644 src/FScript.Compiler/IrArtifact.fs create mode 100644 tests/FScript.Compiler.Tests/IrArtifactTests.fs diff --git a/CHANGELOG.md b/CHANGELOG.md index 1cf9806..ccfdcc5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added deterministic, versioned JSON `.fsir` serialization and validated deserialization for executable LIR. - Added deterministic Go 1.25 project generation for the initial primitive and curried-function compiler slice. - Added executable LIR lowering and interpretation for primitive, curried-function, application, and conditional programs. - Added a reusable compiler assembly with deterministic typed HIR lowering, validation, diagnostics, and debug output. diff --git a/src/FScript.Compiler/FScript.Compiler.fsproj b/src/FScript.Compiler/FScript.Compiler.fsproj index 630a81a..2c3c34e 100644 --- a/src/FScript.Compiler/FScript.Compiler.fsproj +++ b/src/FScript.Compiler/FScript.Compiler.fsproj @@ -10,6 +10,7 @@ + diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs new file mode 100644 index 0000000..c6ffb1b --- /dev/null +++ b/src/FScript.Compiler/IrArtifact.fs @@ -0,0 +1,181 @@ +namespace FScript.Compiler + +open System +open System.IO +open System.Text +open System.Text.Json +open FScript.Language + +module IrArtifact = + [] + let Format = "fscript-lir" + [] + let Version = 1 + + let private writeOptionString (writer: Utf8JsonWriter) (name: string) (value: string option) = + match value with + | Some text -> writer.WriteString(name, text) + | None -> writer.WriteNull(name) + + let rec private writeType (writer: Utf8JsonWriter) value = + writer.WriteStartObject() + let kind, arguments, name = + match value with + | TUnit -> "unit", [], None | TInt -> "int", [], None | TFloat -> "float", [], None + | TBool -> "bool", [], None | TString -> "string", [], None | TTypeToken -> "type-token", [], None + | TList item -> "list", [ item ], None | TTask item -> "task", [ item ], None + | TOption item -> "option", [ item ], None | TTuple items -> "tuple", items, None + | TMap(key, item) -> "map", [ key; item ], None | TFun(argument, result) -> "function", [ argument; result ], None + | TNamed typeName -> "named", [], Some typeName + | TVar id -> "variable", [], Some(string id) + | TRecord fields -> + writer.WriteString("kind", "record") + writer.WriteStartArray("fields") + for fieldName, fieldType in fields |> Map.toList do + writer.WriteStartObject(); writer.WriteString("name", fieldName); writer.WritePropertyName("type") + writeType writer fieldType; writer.WriteEndObject() + writer.WriteEndArray(); writer.WriteEndObject() + "", [], None + | TUnion(typeName, cases) -> + writer.WriteString("kind", "union"); writer.WriteString("name", typeName) + writer.WriteStartArray("cases") + for caseName, payload in cases |> Map.toList do + writer.WriteStartObject(); writer.WriteString("name", caseName) + match payload with + | Some payloadType -> writer.WritePropertyName("payload"); writeType writer payloadType + | None -> writer.WriteNull("payload") + writer.WriteEndObject() + writer.WriteEndArray(); writer.WriteEndObject() + "", [], None + if kind <> "" then + writer.WriteString("kind", kind) + match name with Some value -> writer.WriteString("name", value) | None -> () + if not arguments.IsEmpty then + writer.WriteStartArray("arguments"); arguments |> List.iter (writeType writer); writer.WriteEndArray() + writer.WriteEndObject() + + let private writePosition (writer: Utf8JsonWriter) value = + writer.WriteStartObject(); writeOptionString writer "file" value.File + writer.WriteNumber("line", value.Line); writer.WriteNumber("column", value.Column); writer.WriteEndObject() + + let private writeSpan (writer: Utf8JsonWriter) value = + writer.WriteStartObject(); writer.WritePropertyName("start"); writePosition writer value.Start + writer.WritePropertyName("end"); writePosition writer value.End; writer.WriteEndObject() + + let private constantName = function + | Lir.Unit -> "unit" | Lir.Int _ -> "int" | Lir.Float _ -> "float" | Lir.Bool _ -> "bool" | Lir.String _ -> "string" + + let rec private writeExpression (writer: Utf8JsonWriter) (value: Lir.Expression) = + writer.WriteStartObject(); writer.WritePropertyName("type"); writeType writer value.Type + writer.WritePropertyName("span"); writeSpan writer value.Span + match value.Node with + | Lir.Constant constant -> + writer.WriteString("kind", "constant"); writer.WriteString("constantKind", constantName constant) + match constant with + | Lir.Unit -> () | Lir.Int v -> writer.WriteNumber("intValue", v) | Lir.Float v -> writer.WriteNumber("floatValue", v) + | Lir.Bool v -> writer.WriteBoolean("boolValue", v) | Lir.String v -> writer.WriteString("stringValue", v) + | Lir.Load id -> writer.WriteString("kind", "load"); writer.WriteNumber("symbol", Lir.symbolNumber id) + | Lir.MakeClosure(id, name, parameterType, body) -> + writer.WriteString("kind", "closure"); writer.WriteNumber("symbol", Lir.symbolNumber id); writer.WriteString("name", name) + writer.WritePropertyName("parameterType"); writeType writer parameterType + writer.WritePropertyName("body"); writeExpression writer body + | Lir.CallClosure(fn, argument) -> + writer.WriteString("kind", "call"); writer.WritePropertyName("function"); writeExpression writer fn + writer.WritePropertyName("argument"); writeExpression writer argument + | Lir.Branch(condition, whenTrue, whenFalse) -> + writer.WriteString("kind", "branch"); writer.WritePropertyName("condition"); writeExpression writer condition + writer.WritePropertyName("whenTrue"); writeExpression writer whenTrue; writer.WritePropertyName("whenFalse"); writeExpression writer whenFalse + | Lir.Operation(operation, arguments) -> + writer.WriteString("kind", "operation"); writer.WriteString("operation", sprintf "%A" operation) + writer.WriteStartArray("arguments"); arguments |> List.iter (writeExpression writer); writer.WriteEndArray() + writer.WriteEndObject() + + let serialize (program: Lir.Program) = + use stream = new MemoryStream() + use writer = new Utf8JsonWriter(stream, JsonWriterOptions(Indented = true)) + writer.WriteStartObject(); writer.WriteString("format", Format); writer.WriteNumber("irVersion", Version) + writer.WriteString("compilerVersion", "0.0.0"); writer.WriteString("languageVersion", "0.0.0") + writer.WriteStartArray("sources"); writer.WriteEndArray(); writer.WriteStartArray("dependencies"); writer.WriteEndArray() + writer.WriteStartObject("program"); writer.WriteStartArray("declarations") + for declaration in program.Declarations do + writer.WriteStartObject(); writer.WriteNumber("id", Lir.symbolNumber declaration.Id); writer.WriteString("name", declaration.Name) + writer.WritePropertyName("type"); writeType writer declaration.Type; writer.WriteBoolean("recursive", declaration.IsRecursive) + writer.WriteBoolean("exported", declaration.IsExported); writer.WritePropertyName("span"); writeSpan writer declaration.Span + writer.WritePropertyName("value"); writeExpression writer declaration.Value; writer.WriteEndObject() + writer.WriteEndArray() + match program.Entry with Some entry -> writer.WritePropertyName("entry"); writeExpression writer entry | None -> writer.WriteNull("entry") + writer.WriteEndObject(); writer.WriteEndObject(); writer.Flush(); stream.ToArray() + + let private property (name: string) (element: JsonElement) = element.GetProperty(name) + let private stringProperty (name: string) (element: JsonElement) = + match (property name element).GetString() with null -> "" | value -> value + let private readPosition (element: JsonElement) = + let fileElement = property "file" element + { File = if fileElement.ValueKind = JsonValueKind.Null then None else fileElement.GetString() |> Option.ofObj + Line = (property "line" element).GetInt32(); Column = (property "column" element).GetInt32() } + let private readSpan (element: JsonElement) = { Start = readPosition (property "start" element); End = readPosition (property "end" element) } + + let rec private readType (element: JsonElement) = + let args () = (property "arguments" element).EnumerateArray() |> Seq.map readType |> Seq.toList + match stringProperty "kind" element with + | "unit" -> TUnit | "int" -> TInt | "float" -> TFloat | "bool" -> TBool | "string" -> TString | "type-token" -> TTypeToken + | "list" -> TList(args().Head) | "task" -> TTask(args().Head) | "option" -> TOption(args().Head) | "tuple" -> TTuple(args()) + | "map" -> let values = args() in TMap(values.[0], values.[1]) + | "function" -> let values = args() in TFun(values.[0], values.[1]) + | "named" -> TNamed(stringProperty "name" element) | "variable" -> TVar(Int32.Parse(stringProperty "name" element)) + | "record" -> + (property "fields" element).EnumerateArray() + |> Seq.map (fun field -> stringProperty "name" field, readType (property "type" field)) |> Map.ofSeq |> TRecord + | "union" -> + let cases = (property "cases" element).EnumerateArray() |> Seq.map (fun item -> + let payload = property "payload" item + stringProperty "name" item, if payload.ValueKind = JsonValueKind.Null then None else Some(readType payload)) |> Map.ofSeq + TUnion(stringProperty "name" element, cases) + | kind -> invalidOp $"Unknown IR type kind '{kind}'." + + let private readOperation = function + | "IntAdd" -> Lir.IntAdd | "FloatAdd" -> Lir.FloatAdd | "IntSubtract" -> Lir.IntSubtract | "FloatSubtract" -> Lir.FloatSubtract + | "IntMultiply" -> Lir.IntMultiply | "FloatMultiply" -> Lir.FloatMultiply | "IntDivide" -> Lir.IntDivide | "FloatDivide" -> Lir.FloatDivide + | "IntRemainder" -> Lir.IntRemainder | "StructuralEqual" -> Lir.StructuralEqual | "NumericLessThan" -> Lir.NumericLessThan + | "NumericGreaterThan" -> Lir.NumericGreaterThan | "NumericLessThanOrEqual" -> Lir.NumericLessThanOrEqual + | "NumericGreaterThanOrEqual" -> Lir.NumericGreaterThanOrEqual | "BooleanAnd" -> Lir.BooleanAnd | "BooleanOr" -> Lir.BooleanOr + | value -> invalidOp $"Unknown LIR operation '{value}'." + + let rec private readExpression (element: JsonElement) : Lir.Expression = + let expressionType = readType (property "type" element) + let span = readSpan (property "span" element) + let node = + match stringProperty "kind" element with + | "constant" -> + Lir.Constant(match stringProperty "constantKind" element with + | "unit" -> Lir.Unit | "int" -> Lir.Int((property "intValue" element).GetInt64()) + | "float" -> Lir.Float((property "floatValue" element).GetDouble()) + | "bool" -> Lir.Bool((property "boolValue" element).GetBoolean()) + | "string" -> Lir.String(stringProperty "stringValue" element) | value -> invalidOp $"Unknown constant '{value}'.") + | "load" -> Lir.Load(Lir.SymbolId((property "symbol" element).GetInt32())) + | "closure" -> Lir.MakeClosure(Lir.SymbolId((property "symbol" element).GetInt32()), stringProperty "name" element, + readType (property "parameterType" element), readExpression (property "body" element)) + | "call" -> Lir.CallClosure(readExpression (property "function" element), readExpression (property "argument" element)) + | "branch" -> Lir.Branch(readExpression (property "condition" element), readExpression (property "whenTrue" element), readExpression (property "whenFalse" element)) + | "operation" -> Lir.Operation(readOperation (stringProperty "operation" element), (property "arguments" element).EnumerateArray() |> Seq.map readExpression |> Seq.toList) + | kind -> invalidOp $"Unknown LIR expression kind '{kind}'." + { Type = expressionType; Span = span; Node = node } + + let deserialize (bytes: byte array) = + try + use document = JsonDocument.Parse(bytes) + let root = document.RootElement + if stringProperty "format" root <> Format then Error [ { Code = "FSA5001"; Stage = Artifact; Message = "Unsupported IR artifact format."; Span = None } ] + elif (property "irVersion" root).GetInt32() <> Version then Error [ { Code = "FSA5002"; Stage = Artifact; Message = "Incompatible IR artifact version."; Span = None } ] + else + let programElement = property "program" root + let declarations = (property "declarations" programElement).EnumerateArray() |> Seq.map (fun (item: JsonElement) -> + ({ Id = Lir.SymbolId((property "id" item).GetInt32()); Name = stringProperty "name" item; Type = readType (property "type" item) + IsRecursive = (property "recursive" item).GetBoolean(); IsExported = (property "exported" item).GetBoolean() + Span = readSpan (property "span" item); Value = readExpression (property "value" item) }: Lir.Declaration)) |> Seq.toList + let entryElement = property "entry" programElement + let program : Lir.Program = + { Declarations = declarations + Entry = if entryElement.ValueKind = JsonValueKind.Null then None else Some(readExpression entryElement) } + LirValidation.validate program + with ex -> Error [ { Code = "FSA5003"; Stage = Artifact; Message = ex.Message; Span = None } ] diff --git a/tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj b/tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj index 1c23861..f8317e3 100644 --- a/tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj +++ b/tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj @@ -6,6 +6,7 @@ + diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs new file mode 100644 index 0000000..ec6dc96 --- /dev/null +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -0,0 +1,41 @@ +namespace FScript.Compiler.Tests + +open System +open System.Text +open FScript.Compiler +open FScript.Language +open NUnit.Framework + +[] +type IrArtifactTests() = + let compile source = + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Ok value -> value + | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``IR artifact is deterministic and executable after round trip`` () = + let compiled = compile "let add x y = x + y\nadd 1 2" + let first = IrArtifact.serialize compiled.Lir + let second = IrArtifact.serialize compiled.Lir + Assert.That((first = second), Is.True) + match IrArtifact.deserialize first with + | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) + | Ok loaded -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } loaded with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 3L)) + | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``IR artifact rejects incompatible versions`` () = + let bytes = compile "1" |> _.Lir |> IrArtifact.serialize + let invalid = Encoding.UTF8.GetString(bytes).Replace("\"irVersion\": 1", "\"irVersion\": 999") |> Encoding.UTF8.GetBytes + match IrArtifact.deserialize invalid with + | Ok _ -> Assert.Fail("Expected incompatible-version diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSA5002")) + + [] + member _.``IR artifact rejects corrupt JSON`` () = + match IrArtifact.deserialize (Encoding.UTF8.GetBytes("{")) with + | Ok _ -> Assert.Fail("Expected corrupt-artifact diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSA5003")) From 429cb446a3d6c8d543a3e2b0854ad1c66666a47a Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 16:38:49 +0200 Subject: [PATCH 007/133] feat(cli): add IR and Go compile targets --- CHANGELOG.md | 1 + docs/architecture/assemblies-and-roles.md | 12 ++++ src/FScript.Compiler/Diagnostics.fs | 2 +- src/FScript.Compiler/GoBackend.fs | 6 +- src/FScript/FScript.fsproj | 1 + src/FScript/Program.fs | 69 ++++++++++++++++++++--- tests/FScript.Cli.Tests/CliTests.fs | 33 +++++++++++ website/docs/reference/cli-reference.md | 16 ++++++ 8 files changed, 129 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ccfdcc5..0a7531a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added `fscript compile` targets for whole-program `.fsir` artifacts and deterministic Go source projects. - Added deterministic, versioned JSON `.fsir` serialization and validated deserialization for executable LIR. - Added deterministic Go 1.25 project generation for the initial primitive and curried-function compiler slice. - Added executable LIR lowering and interpretation for primitive, curried-function, application, and conditional programs. diff --git a/docs/architecture/assemblies-and-roles.md b/docs/architecture/assemblies-and-roles.md index 41b0753..4b68306 100644 --- a/docs/architecture/assemblies-and-roles.md +++ b/docs/architecture/assemblies-and-roles.md @@ -38,6 +38,18 @@ Use this when: - You want to embed the language in another .NET application. - You need language processing independently from default runtime extern catalog. +### `FScript.Compiler` +Role: +- Reusable .NET compiler pipeline for typed HIR, executable LIR, IR artifacts, interpretation, and source backends. + +Responsibilities: +- Consume the resolved typed result produced by `FScript.Language`. +- Validate and lower HIR to backend-independent LIR. +- Serialize and validate whole-program `.fsir` artifacts. +- Execute supported LIR and generate deterministic Go source projects. + +`FScript.Compiler` depends on `FScript.Language` and is intentionally excluded from the Fable JavaScript package. + NuGet: - `MagnusOpera.FScript.Language` diff --git a/src/FScript.Compiler/Diagnostics.fs b/src/FScript.Compiler/Diagnostics.fs index e6f612a..131b676 100644 --- a/src/FScript.Compiler/Diagnostics.fs +++ b/src/FScript.Compiler/Diagnostics.fs @@ -6,7 +6,7 @@ type DiagnosticStage = | Hir | Lowering | Interpreter - | GoBackend + | GoGeneration | Artifact type CompilerDiagnostic = diff --git a/src/FScript.Compiler/GoBackend.fs b/src/FScript.Compiler/GoBackend.fs index 8d8b6de..eb67bd3 100644 --- a/src/FScript.Compiler/GoBackend.fs +++ b/src/FScript.Compiler/GoBackend.fs @@ -91,9 +91,9 @@ module GoBackend = | Some value -> emitExpression names value match error, entry with | Some(message, span), _ -> - Error [ { Code = "FSB4001"; Stage = GoBackend; Message = message; Span = Some span } ] + Error [ { Code = "FSB4001"; Stage = GoGeneration; Message = message; Span = Some span } ] | None, Error message -> - Error [ { Code = "FSB4001"; Stage = GoBackend; Message = message; Span = None } ] + Error [ { Code = "FSB4001"; Stage = GoGeneration; Message = message; Span = None } ] | None, Ok entryCode -> let moduleName = if String.IsNullOrWhiteSpace options.ModuleName then "fscript.generated/program" else options.ModuleName let programSource = $"package fscriptprogram\n\nimport \"{moduleName}/internal/fsruntime\"\n\n{declarations}\nfunc Run() any {{ return {entryCode} }}\n" @@ -103,4 +103,4 @@ module GoBackend = { RelativePath = "internal/fscriptprogram/program.go"; Content = programSource } { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nfunc Int(value any) int64 { return value.(int64) }\n" } ] with ex -> - Error [ { Code = "FSB4002"; Stage = GoBackend; Message = ex.Message; Span = None } ] + Error [ { Code = "FSB4002"; Stage = GoGeneration; Message = ex.Message; Span = None } ] diff --git a/src/FScript/FScript.fsproj b/src/FScript/FScript.fsproj index e600a17..2daeb1d 100644 --- a/src/FScript/FScript.fsproj +++ b/src/FScript/FScript.fsproj @@ -39,6 +39,7 @@ + diff --git a/src/FScript/Program.fs b/src/FScript/Program.fs index 23e9fe3..928e503 100644 --- a/src/FScript/Program.fs +++ b/src/FScript/Program.fs @@ -7,6 +7,58 @@ open CliArgs open FScript.Language open FScript.Runtime +let writeGeneratedFile (root: string) (file: FScript.Compiler.GeneratedFile) = + let path = Path.Combine(root, file.RelativePath) + match Path.GetDirectoryName(path) with + | null -> () + | directory -> Directory.CreateDirectory(directory) |> ignore + File.WriteAllText(path, file.Content) + +let runCompileCommand (argv: string array) = + let usage () = + Console.Error.WriteLine("Usage: fscript compile --target --output ") + 1 + if argv.Length < 2 then usage () + else + let scriptPath = Path.GetFullPath(argv.[1]) + let tryOption name = + argv |> Array.tryFindIndex ((=) name) + |> Option.bind (fun index -> if index + 1 < argv.Length then Some argv.[index + 1] else None) + match tryOption "--target", tryOption "--output" with + | Some target, Some output when File.Exists scriptPath -> + try + let outputDirectory = Path.GetFullPath(output) + let sourceRoot = Path.GetDirectoryName(scriptPath) |> Option.ofObj |> Option.defaultValue (Directory.GetCurrentDirectory()) + let program = FScript.parseFileWithIncludes sourceRoot scriptPath + match FScript.Compiler.Compiler.compile { Program = program; Externs = [] } with + | Error diagnostics -> + diagnostics |> List.iter (fun diagnostic -> Console.Error.WriteLine($"{diagnostic.Code}: {diagnostic.Message}")) + 4 + | Ok compiled -> + Directory.CreateDirectory(outputDirectory) |> ignore + match target.ToLowerInvariant() with + | "ir" -> + let baseName = + match Path.GetFileNameWithoutExtension(scriptPath) with + | null | "" -> "program" + | value -> value + let name = baseName + ".fsir" + File.WriteAllBytes(Path.Combine(outputDirectory, name), FScript.Compiler.IrArtifact.serialize compiled.Lir) + 0 + | "go" -> + match FScript.Compiler.GoBackend.generate { ModuleName = "fscript.generated/program" } compiled.Lir with + | Error diagnostics -> + diagnostics |> List.iter (fun diagnostic -> Console.Error.WriteLine($"{diagnostic.Code}: {diagnostic.Message}")) + 4 + | Ok files -> files |> List.iter (writeGeneratedFile outputDirectory); 0 + | _ -> usage () + with + | ParseException error -> Console.Error.WriteLine($"Parse error: {error.Message}"); 1 + | TypeException error -> Console.Error.WriteLine($"Type error: {error.Message}"); 2 + | ex -> Console.Error.WriteLine(ex.Message); 4 + | Some _, Some _ -> Console.Error.WriteLine($"File not found: {scriptPath}"); 1 + | _ -> usage () + let formatSpan (span: Span) = match span.Start.File with | Some file -> sprintf "(%s:%d:%d)" file span.Start.Line span.Start.Column @@ -375,9 +427,12 @@ let runRepl (externs: ExternalFunction list) (_rootDirectory: string) = [] let main argv = - let parser = ArgumentParser.Create(programName = "fscript") - let cliArgv, scriptArguments = splitScriptArguments argv - try + if argv.Length > 0 && String.Equals(argv.[0], "compile", StringComparison.OrdinalIgnoreCase) then + runCompileCommand argv + else + let parser = ArgumentParser.Create(programName = "fscript") + let cliArgv, scriptArguments = splitScriptArguments argv + try if scriptArguments.IsEmpty && cliArgv.Length = 1 && String.Equals(cliArgv.[0], "version", StringComparison.OrdinalIgnoreCase) then Console.WriteLine(printVersion ()) 0 @@ -433,7 +488,7 @@ let main argv = | EvalException err -> Console.Error.WriteLine($"Eval error {formatSpan err.Span}: {err.Message}") 3 - with - | :? ArguParseException as ex -> - Console.Error.WriteLine(ex.Message) - 1 + with + | :? ArguParseException as ex -> + Console.Error.WriteLine(ex.Message) + 1 diff --git a/tests/FScript.Cli.Tests/CliTests.fs b/tests/FScript.Cli.Tests/CliTests.fs index 1b5758d..ac7fdda 100644 --- a/tests/FScript.Cli.Tests/CliTests.fs +++ b/tests/FScript.Cli.Tests/CliTests.fs @@ -109,6 +109,39 @@ type CliTests() = Assert.That(code, Is.EqualTo(0), $"stderr: {stderr}") Assert.That(Regex.IsMatch(stdout.Trim(), "^\\d+\\.\\d+\\.\\d+"), Is.True) + [] + member _.``Compiles a whole program to versioned IR`` () = + let repoRoot = findRepoRoot () + let tempRoot = Path.Combine(Path.GetTempPath(), $"fscript-compile-ir-{Guid.NewGuid():N}") + let script = Path.Combine(tempRoot, "program.fss") + let output = Path.Combine(tempRoot, "out") + try + Directory.CreateDirectory(tempRoot) |> ignore + File.WriteAllText(script, "let add x y = x + y\nadd 1 2") + let code, _, stderr = runCli repoRoot repoRoot [ "compile"; script; "--target"; "ir"; "--output"; output ] None + Assert.That(code, Is.EqualTo(0), stderr) + let artifact = Path.Combine(output, "program.fsir") + Assert.That(File.Exists artifact, Is.True) + Assert.That(File.ReadAllText(artifact), Does.Contain("\"format\": \"fscript-lir\"")) + finally + if Directory.Exists tempRoot then Directory.Delete(tempRoot, true) + + [] + member _.``Compiles a whole program to a Go project`` () = + let repoRoot = findRepoRoot () + let tempRoot = Path.Combine(Path.GetTempPath(), $"fscript-compile-go-{Guid.NewGuid():N}") + let script = Path.Combine(tempRoot, "program.fss") + let output = Path.Combine(tempRoot, "out") + try + Directory.CreateDirectory(tempRoot) |> ignore + File.WriteAllText(script, "let add x y = x + y\nadd 1 2") + let code, _, stderr = runCli repoRoot repoRoot [ "compile"; script; "--target"; "go"; "--output"; output ] None + Assert.That(code, Is.EqualTo(0), stderr) + Assert.That(File.Exists(Path.Combine(output, "go.mod")), Is.True) + Assert.That(File.Exists(Path.Combine(output, "internal", "fscriptprogram", "program.go")), Is.True) + finally + if Directory.Exists tempRoot then Directory.Delete(tempRoot, true) + [] member _.``No args with redirected stdin executes stdin mode`` () = let repoRoot = findRepoRoot () diff --git a/website/docs/reference/cli-reference.md b/website/docs/reference/cli-reference.md index 48db5b4..c5af436 100644 --- a/website/docs/reference/cli-reference.md +++ b/website/docs/reference/cli-reference.md @@ -20,6 +20,22 @@ fscript fscript version ``` +## Compile programs + +Generate one versioned whole-program IR artifact: + +```bash +fscript compile program.fss --target ir --output ./out +``` + +Generate an inspectable Go 1.25 source project without invoking `go build`: + +```bash +fscript compile program.fss --target go --output ./out +``` + +The IR target writes `out/program.fsir`. The Go target writes `go.mod`, `main.go`, generated program source, and the required FScript runtime package. Unsupported features use structured compiler/backend diagnostic codes. + ## From source ```bash From 0a95e42543f9087ecaa1ccd1e6a805f823dc405d Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 16:44:08 +0200 Subject: [PATCH 008/133] feat(compiler): execute aggregate LIR values --- CHANGELOG.md | 1 + docs/specs/compiler-semantics-baseline.md | 2 + src/FScript.Compiler/GoBackend.fs | 57 +++++++++++++++---- src/FScript.Compiler/Hir.fs | 19 +++++++ src/FScript.Compiler/HirLowering.fs | 54 +++++++++++++++--- src/FScript.Compiler/HirValidation.fs | 5 ++ src/FScript.Compiler/IrArtifact.fs | 30 ++++++++++ src/FScript.Compiler/Lir.fs | 16 ++++++ src/FScript.Compiler/LirInterpreter.fs | 34 +++++++++++ .../FScript.Compiler.Tests/GoBackendTests.fs | 10 ++-- tests/FScript.Compiler.Tests/HirTests.fs | 22 +++++-- .../FScript.Compiler.Tests/IrArtifactTests.fs | 4 +- 12 files changed, 224 insertions(+), 30 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0a7531a..1fe75ae 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added compiler, serialized LIR, interpreter, and Go support for tuples, records, options, lists, append, and recursive structural equality. - Added `fscript compile` targets for whole-program `.fsir` artifacts and deterministic Go source projects. - Added deterministic, versioned JSON `.fsir` serialization and validated deserialization for executable LIR. - Added deterministic Go 1.25 project generation for the initial primitive and curried-function compiler slice. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index d6359e4..ad8140d 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -10,6 +10,8 @@ This document records the behavior that the typed IR, LIR interpreter, and compi - Hindley-Milner inference generalizes top-level and local let bindings after applying substitutions. Recursive bindings are inferred monomorphically inside their recursive group and generalized afterward. - The active typed program stores top-level types but retains parser expressions in bodies. Compiler lowering therefore requires a new resolved, recursively typed result emitted by the inference pass; types cannot safely be reconstructed afterward. - Top-level statements execute in source order. Every non-final expression must have type `unit`; the final expression is the program result. + +The first aggregate compiler slice preserves tuple field order, record field names, option presence, list element order, list cons/append behavior, and recursively structural equality through HIR, serialized LIR, interpretation, and Go generation. - Functions are curried. Multi-argument declarations are nested lambdas, application is one argument at a time, and closures capture their lexical environment. Single and mutual recursive function groups are supported. ## Types and values diff --git a/src/FScript.Compiler/GoBackend.fs b/src/FScript.Compiler/GoBackend.fs index eb67bd3..1652597 100644 --- a/src/FScript.Compiler/GoBackend.fs +++ b/src/FScript.Compiler/GoBackend.fs @@ -16,6 +16,10 @@ module GoBackend = | TString -> "string" | TVar _ -> "any" | TFun(argument, result) -> $"func({goType argument}) {goType result}" + | TTuple _ -> "[]any" + | TList _ -> "[]any" + | TOption _ -> "fsruntime.Option" + | TRecord _ -> "map[string]any" | value -> invalidArg "value" $"Unsupported Go representation for {Types.typeToString value}." let private identifier prefix id = $"{prefix}{Lir.symbolNumber id}" @@ -34,7 +38,7 @@ module GoBackend = | Lir.IntMultiply | Lir.FloatMultiply -> Some "*" | Lir.IntDivide | Lir.FloatDivide -> Some "/" | Lir.IntRemainder -> Some "%" - | Lir.StructuralEqual -> Some "==" + | Lir.StructuralEqual -> None | Lir.NumericLessThan -> Some "<" | Lir.NumericGreaterThan -> Some ">" | Lir.NumericLessThanOrEqual -> Some "<=" @@ -62,18 +66,47 @@ module GoBackend = | Ok c, Ok t, Ok f -> Ok $"func() {goType expression.Type} {{ if {c} {{ return {t} }}; return {f} }}()" | Error error, _, _ | _, Error error, _ | _, _, Error error -> Error error | Lir.Operation(op, [ left; right ]) -> - match operation op, emitExpression environment left, emitExpression environment right with - | Some token, Ok l, Ok r -> - let l, r = - match op with - | Lir.IntAdd | Lir.IntSubtract | Lir.IntMultiply | Lir.IntDivide | Lir.IntRemainder -> - $"fsruntime.Int({l})", $"fsruntime.Int({r})" - | _ -> l, r - Ok $"({l} {token} {r})" - | None, _, _ -> Error $"Unsupported Go operation {op}" - | _, Error error, _ | _, _, Error error -> Error error + match emitExpression environment left, emitExpression environment right with + | Ok l, Ok r when op = Lir.StructuralEqual -> Ok $"fsruntime.Equal({l}, {r})" + | Ok l, Ok r -> + match operation op with + | Some token -> + let l, r = + match op with + | Lir.IntAdd | Lir.IntSubtract | Lir.IntMultiply | Lir.IntDivide | Lir.IntRemainder -> + $"fsruntime.Int({l})", $"fsruntime.Int({r})" + | _ -> l, r + Ok $"({l} {token} {r})" + | None -> Error $"Unsupported Go operation {op}" + | Error error, _ | _, Error error -> Error error + | Lir.Tuple items | Lir.List items -> + items |> List.map (emitExpression environment) |> collect + |> Result.map (fun values -> + let joined = String.concat ", " values + $"[]any{{{joined}}}") + | Lir.Record fields -> + fields |> List.map (fun (name, value) -> emitExpression environment value |> Result.map (fun code -> $"{System.Text.Json.JsonSerializer.Serialize(name)}: {code}")) + |> collect |> Result.map (fun values -> + let joined = String.concat ", " values + $"map[string]any{{{joined}}}") + | Lir.FieldGet(target, name) -> + emitExpression environment target |> Result.map (fun code -> $"({code})[{System.Text.Json.JsonSerializer.Serialize(name)}]") + | Lir.SomeValue item -> emitExpression environment item |> Result.map (fun code -> $"fsruntime.Some({code})") + | Lir.NoneValue -> Ok "fsruntime.None()" + | Lir.Cons(head, tail) -> + match emitExpression environment head, emitExpression environment tail with + | Ok h, Ok t -> Ok $"fsruntime.Cons({h}, {t})" + | Error error, _ | _, Error error -> Error error + | Lir.Append(left, right) -> + match emitExpression environment left, emitExpression environment right with + | Ok l, Ok r -> Ok $"fsruntime.Append({l}, {r})" + | Error error, _ | _, Error error -> Error error | Lir.Operation _ -> Error "Invalid operation arity" + and private collect<'a> (results: Result<'a, string> list) : Result<'a list, string> = + results |> List.fold (fun state result -> + match state, result with Ok values, Ok value -> Ok(values @ [ value ]) | Error error, _ | _, Error error -> Error error) (Ok []) + let generate options (program: Lir.Program) : Result = try let names = @@ -101,6 +134,6 @@ module GoBackend = [ { RelativePath = "go.mod"; Content = $"module {moduleName}\n\ngo 1.25\n" } { RelativePath = "main.go"; Content = $"package main\n\nimport (\n\t\"fmt\"\n\t\"{moduleName}/internal/fscriptprogram\"\n)\n\nfunc main() {{ fmt.Println(fscriptprogram.Run()) }}\n" } { RelativePath = "internal/fscriptprogram/program.go"; Content = programSource } - { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nfunc Int(value any) int64 { return value.(int64) }\n" } ] + { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nimport \"reflect\"\n\ntype Option struct { HasValue bool; Value any }\n\nfunc Int(value any) int64 { return value.(int64) }\nfunc Some(value any) Option { return Option{HasValue: true, Value: value} }\nfunc None() Option { return Option{} }\nfunc Equal(left any, right any) bool { return reflect.DeepEqual(left, right) }\nfunc Cons(head any, tail []any) []any { return append([]any{head}, tail...) }\nfunc Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) }\n" } ] with ex -> Error [ { Code = "FSB4002"; Stage = GoGeneration; Message = ex.Message; Span = None } ] diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs index f43a82b..a0b4fac 100644 --- a/src/FScript.Compiler/Hir.fs +++ b/src/FScript.Compiler/Hir.fs @@ -42,6 +42,14 @@ module Hir = | Apply of Expression * Expression | If of Expression * Expression * Expression | Primitive of PrimitiveOperation * Expression list + | Tuple of Expression list + | List of Expression list + | Record of (string * Expression) list + | FieldGet of Expression * string + | SomeValue of Expression + | NoneValue + | Cons of Expression * Expression + | Append of Expression * Expression type Declaration = { Id: SymbolId @@ -93,6 +101,17 @@ module Hir = | Primitive(operation, arguments) -> lines.Add(typed $"primitive {formatPrimitive operation}") arguments |> List.iter (expression (indent + 2)) + | Tuple items | List items -> + lines.Add(typed (match value.Node with Tuple _ -> "tuple" | _ -> "list")) + items |> List.iter (expression (indent + 2)) + | Record fields -> + lines.Add(typed "record") + fields |> List.iter (fun (name, item) -> lines.Add($"{prefix} field {name}"); expression (indent + 4) item) + | FieldGet(target, name) -> lines.Add(typed $"field-get {name}"); expression (indent + 2) target + | SomeValue item -> lines.Add(typed "some"); expression (indent + 2) item + | NoneValue -> lines.Add(typed "none") + | Cons(head, tail) -> lines.Add(typed "cons"); expression (indent + 2) head; expression (indent + 2) tail + | Append(left, right) -> lines.Add(typed "append"); expression (indent + 2) left; expression (indent + 2) right for declaration in program.Declarations do lines.Add(sprintf "decl %%%d %s: %s" (symbolNumber declaration.Id) declaration.Name (formatType (let (Forall(_, value)) = declaration.Scheme in value))) diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index d2ad521..fe9b225 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -97,16 +97,54 @@ module HirLowering = match capturedOperandType, inferred with | TVar _, (TInt | TFloat as concrete) -> concrete | value, _ -> value - match operation op operandType with - | None -> Error [ Diagnostic.hir "FSH1003" $"Unresolved primitive operation '{op}'." (Some span) ] - | Some primitive -> - match lowerExpression state environment operandType left, lowerExpression state environment operandType right with - | Ok l, Ok r -> Ok(make (Hir.Primitive(primitive, [ l; r ]))) - | Error l, Error r -> Error(l @ r) - | Error errors, _ - | _, Error errors -> Error errors + if op = "@" then + match lowerExpression state environment inferred left, lowerExpression state environment inferred right with + | Ok l, Ok r -> Ok(make (Hir.Append(l, r))) + | Error a, Error b -> Error(a @ b) + | Error errors, _ | _, Error errors -> Error errors + else + match operation op operandType with + | None -> Error [ Diagnostic.hir "FSH1003" $"Unresolved primitive operation '{op}'." (Some span) ] + | Some primitive -> + match lowerExpression state environment operandType left, lowerExpression state environment operandType right with + | Ok l, Ok r -> Ok(make (Hir.Primitive(primitive, [ l; r ]))) + | Error l, Error r -> Error(l @ r) + | Error errors, _ + | _, Error errors -> Error errors + | ETuple(items, _) -> + let itemTypes = match inferred with TTuple values -> values | _ -> List.replicate items.Length (Types.freshVar()) + List.zip items itemTypes |> List.map (fun (item, itemType) -> lowerExpression state environment itemType item) + |> collectResults |> Result.map (Hir.Tuple >> make) + | EList(items, _) -> + let itemType = match inferred with TList value -> value | _ -> Types.freshVar() + items |> List.map (lowerExpression state environment itemType) |> collectResults |> Result.map (Hir.List >> make) + | ERecord(fields, _) | EStructuralRecord(fields, _) -> + let fieldTypes = match inferred with TRecord values -> values | _ -> Map.empty + fields |> List.map (fun (name, item) -> lowerExpression state environment (fieldTypes |> Map.tryFind name |> Option.defaultValue (Types.freshVar())) item |> Result.map (fun value -> name, value)) + |> collectResults |> Result.map (Hir.Record >> make) + | EFieldGet(target, name, _) -> + lowerExpression state environment (expressionType state target (TRecord Map.empty)) target |> Result.map (fun value -> make (Hir.FieldGet(value, name))) + | ESome(item, _) -> + let itemType = match inferred with TOption value -> value | _ -> Types.freshVar() + lowerExpression state environment itemType item |> Result.map (Hir.SomeValue >> make) + | ENone _ -> Ok(make Hir.NoneValue) + | ECons(head, tail, _) -> + let itemType = match inferred with TList value -> value | _ -> Types.freshVar() + match lowerExpression state environment itemType head, lowerExpression state environment inferred tail with + | Ok h, Ok t -> Ok(make (Hir.Cons(h, t))) | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e + | EAppend(left, right, _) -> + match lowerExpression state environment inferred left, lowerExpression state environment inferred right with + | Ok l, Ok r -> Ok(make (Hir.Append(l, r))) | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e | _ -> Error [ Diagnostic.hir "FSH1099" $"HIR lowering does not yet support {expression.GetType().Name}." (Some span) ] + and private collectResults<'a> (results: Result<'a, CompilerDiagnostic list> list) : Result<'a list, CompilerDiagnostic list> = + let folder state result = + match state, result with + | Ok values, Ok value -> Ok(values @ [ value ]) + | Error a, Error b -> Error(a @ b) + | Error errors, _ | _, Error errors -> Error errors + results |> List.fold folder (Ok []) + let lower (resolved: TypeInfer.ResolvedProgram) : Result = let expressionTypes = resolved.ExpressionTypes diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index d06b4f4..706e160 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -16,6 +16,11 @@ module HirValidation = | Hir.If(condition, whenTrue, whenFalse) -> Set.unionMany [ references condition; references whenTrue; references whenFalse ] | Hir.Primitive(_, arguments) -> arguments |> List.map references |> Set.unionMany + | Hir.Tuple items | Hir.List items -> items |> List.map references |> Set.unionMany + | Hir.Record fields -> fields |> List.map (snd >> references) |> Set.unionMany + | Hir.FieldGet(target, _) | Hir.SomeValue target -> references target + | Hir.NoneValue -> Set.empty + | Hir.Cons(head, tail) | Hir.Append(head, tail) -> Set.union (references head) (references tail) let declared = declarationIds |> Set.ofList let unresolved = [ yield! program.Declarations |> List.collect (fun value -> references value.Value |> Set.toList) diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index c6ffb1b..ebf2cc6 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -88,6 +88,27 @@ module IrArtifact = | Lir.Operation(operation, arguments) -> writer.WriteString("kind", "operation"); writer.WriteString("operation", sprintf "%A" operation) writer.WriteStartArray("arguments"); arguments |> List.iter (writeExpression writer); writer.WriteEndArray() + | Lir.Tuple items | Lir.List items -> + writer.WriteString("kind", match value.Node with Lir.Tuple _ -> "tuple" | _ -> "list") + writer.WriteStartArray("items"); items |> List.iter (writeExpression writer); writer.WriteEndArray() + | Lir.Record fields -> + writer.WriteString("kind", "record"); writer.WriteStartArray("fields") + for name, field in fields do + writer.WriteStartObject(); writer.WriteString("name", name); writer.WritePropertyName("value") + writeExpression writer field; writer.WriteEndObject() + writer.WriteEndArray() + | Lir.FieldGet(target, name) -> + writer.WriteString("kind", "field-get"); writer.WriteString("name", name) + writer.WritePropertyName("target"); writeExpression writer target + | Lir.SomeValue item -> + writer.WriteString("kind", "some"); writer.WritePropertyName("value"); writeExpression writer item + | Lir.NoneValue -> writer.WriteString("kind", "none") + | Lir.Cons(head, tail) -> + writer.WriteString("kind", "cons"); writer.WritePropertyName("head"); writeExpression writer head + writer.WritePropertyName("tail"); writeExpression writer tail + | Lir.Append(left, right) -> + writer.WriteString("kind", "append"); writer.WritePropertyName("left"); writeExpression writer left + writer.WritePropertyName("right"); writeExpression writer right writer.WriteEndObject() let serialize (program: Lir.Program) = @@ -158,6 +179,15 @@ module IrArtifact = | "call" -> Lir.CallClosure(readExpression (property "function" element), readExpression (property "argument" element)) | "branch" -> Lir.Branch(readExpression (property "condition" element), readExpression (property "whenTrue" element), readExpression (property "whenFalse" element)) | "operation" -> Lir.Operation(readOperation (stringProperty "operation" element), (property "arguments" element).EnumerateArray() |> Seq.map readExpression |> Seq.toList) + | "tuple" -> Lir.Tuple((property "items" element).EnumerateArray() |> Seq.map readExpression |> Seq.toList) + | "list" -> Lir.List((property "items" element).EnumerateArray() |> Seq.map readExpression |> Seq.toList) + | "record" -> + Lir.Record((property "fields" element).EnumerateArray() |> Seq.map (fun field -> stringProperty "name" field, readExpression (property "value" field)) |> Seq.toList) + | "field-get" -> Lir.FieldGet(readExpression (property "target" element), stringProperty "name" element) + | "some" -> Lir.SomeValue(readExpression (property "value" element)) + | "none" -> Lir.NoneValue + | "cons" -> Lir.Cons(readExpression (property "head" element), readExpression (property "tail" element)) + | "append" -> Lir.Append(readExpression (property "left" element), readExpression (property "right" element)) | kind -> invalidOp $"Unknown LIR expression kind '{kind}'." { Type = expressionType; Span = span; Node = node } diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index 458913d..7221845 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -30,6 +30,14 @@ module Lir = | CallClosure of Expression * Expression | Branch of Expression * Expression * Expression | Operation of Operation * Expression list + | Tuple of Expression list + | List of Expression list + | Record of (string * Expression) list + | FieldGet of Expression * string + | SomeValue of Expression + | NoneValue + | Cons of Expression * Expression + | Append of Expression * Expression type Declaration = { Id: SymbolId @@ -73,6 +81,14 @@ module LirLowering = | Hir.Apply(fn, argument) -> Lir.CallClosure(expression fn, expression argument) | Hir.If(condition, whenTrue, whenFalse) -> Lir.Branch(expression condition, expression whenTrue, expression whenFalse) | Hir.Primitive(op, arguments) -> Lir.Operation(operation op, arguments |> List.map expression) + | Hir.Tuple items -> Lir.Tuple(items |> List.map expression) + | Hir.List items -> Lir.List(items |> List.map expression) + | Hir.Record fields -> Lir.Record(fields |> List.map (fun (name, item) -> name, expression item)) + | Hir.FieldGet(target, name) -> Lir.FieldGet(expression target, name) + | Hir.SomeValue item -> Lir.SomeValue(expression item) + | Hir.NoneValue -> Lir.NoneValue + | Hir.Cons(head, tail) -> Lir.Cons(expression head, expression tail) + | Hir.Append(left, right) -> Lir.Append(expression left, expression right) { Type = value.Type; Span = value.Span; Node = node } let lower (program: Hir.Program) : Lir.Program = diff --git a/src/FScript.Compiler/LirInterpreter.fs b/src/FScript.Compiler/LirInterpreter.fs index 2d5bd61..bf2027f 100644 --- a/src/FScript.Compiler/LirInterpreter.fs +++ b/src/FScript.Compiler/LirInterpreter.fs @@ -12,6 +12,10 @@ type LirValue = | VFloat of float | VBool of bool | VString of string + | VTuple of LirValue list + | VList of LirValue list + | VRecord of Map + | VOption of LirValue option | VClosure of Lir.SymbolId * string * Lir.Expression * Map ref type ExecutionResult = { Value: LirValue } @@ -24,6 +28,10 @@ module LirInterpreter = | VFloat a, VFloat b -> a = b | VBool a, VBool b -> a = b | VString a, VString b -> a = b + | VTuple a, VTuple b | VList a, VList b -> a.Length = b.Length && List.forall2 equal a b + | VRecord a, VRecord b -> a.Count = b.Count && (a |> Map.forall (fun key value -> b |> Map.tryFind key |> Option.exists (equal value))) + | VOption None, VOption None -> true + | VOption(Some a), VOption(Some b) -> equal a b | _ -> false let private fail code span message = @@ -79,6 +87,32 @@ module LirInterpreter = | Lir.BooleanAnd, VBool a, VBool b -> Ok(VBool(a && b)) | Lir.BooleanOr, VBool a, VBool b -> Ok(VBool(a || b)) | _ -> fail "FSI3006" expression.Span "Primitive operands do not match their LIR operation.") + | Lir.Tuple items -> evaluateMany environment items |> Result.map VTuple + | Lir.List items -> evaluateMany environment items |> Result.map VList + | Lir.Record fields -> + fields |> List.map (fun (name, item) -> evaluate environment item |> Result.map (fun value -> name, value)) + |> collect |> Result.map (Map.ofList >> VRecord) + | Lir.FieldGet(target, name) -> + match evaluate environment target with + | Ok(VRecord fields) -> fields |> Map.tryFind name |> Option.map Ok |> Option.defaultWith (fun () -> fail "FSI3007" expression.Span $"Record field '{name}' was not found.") + | Ok _ -> fail "FSI3007" expression.Span "Field access requires a record." + | Error errors -> Error errors + | Lir.SomeValue item -> evaluate environment item |> Result.map (Some >> VOption) + | Lir.NoneValue -> Ok(VOption None) + | Lir.Cons(head, tail) -> + match evaluate environment head, evaluate environment tail with + | Ok h, Ok(VList t) -> Ok(VList(h :: t)) | Ok _, Ok _ -> fail "FSI3008" expression.Span "Cons tail must be a list." + | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e + | Lir.Append(left, right) -> + match evaluate environment left, evaluate environment right with + | Ok(VList l), Ok(VList r) -> Ok(VList(l @ r)) | Ok _, Ok _ -> fail "FSI3009" expression.Span "Append operands must be lists." + | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e + + and private evaluateMany environment items = items |> List.map (evaluate environment) |> collect + + and private collect<'a> (results: Result<'a, CompilerDiagnostic list> list) : Result<'a list, CompilerDiagnostic list> = + results |> List.fold (fun state result -> + match state, result with Ok values, Ok value -> Ok(values @ [ value ]) | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e) (Ok []) let execute (_context: InterpreterContext) (program: Lir.Program) = let environment = ref Map.empty diff --git a/tests/FScript.Compiler.Tests/GoBackendTests.fs b/tests/FScript.Compiler.Tests/GoBackendTests.fs index d893a5d..e7e8073 100644 --- a/tests/FScript.Compiler.Tests/GoBackendTests.fs +++ b/tests/FScript.Compiler.Tests/GoBackendTests.fs @@ -9,9 +9,11 @@ open NUnit.Framework [] type GoBackendTests() = - [] - member _.``Generated add project runs with Go`` () = - let input = { Program = Parser.parseProgram "let add x y = x + y\nadd 1 2"; Externs = [] } + [] + [] + [] + member _.``Generated projects run with Go`` (source: string, expected: string) = + let input = { Program = Parser.parseProgram source; Externs = [] } let compiled = match Compiler.compile input with | Ok value -> value @@ -42,6 +44,6 @@ type GoBackendTests() = let error = proc.StandardError.ReadToEnd() proc.WaitForExit() Assert.That(proc.ExitCode, Is.EqualTo(0), error) - Assert.That(output.Trim(), Is.EqualTo("3")) + Assert.That(output.Trim(), Is.EqualTo(expected)) finally if Directory.Exists directory then Directory.Delete(directory, true) diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index a47443d..3023938 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -23,8 +23,22 @@ type HirTests() = | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) [] - member _.``Reports unsupported HIR forms as diagnostics`` () = - let input = { Program = Parser.parseProgram "[1; 2]"; Externs = [] } + member _.``Executes aggregate expressions in LIR`` () = + let input = { Program = Parser.parseProgram "[1] @ [2; 3]"; Externs = [] } match Compiler.compile input with - | Ok _ -> Assert.Fail("Expected unsupported-form diagnostic") - | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSH1099")) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VList [ LirValue.VInt 1L; LirValue.VInt 2L; LirValue.VInt 3L ])) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + [] + [] + member _.``Lowers tuples options and records`` (source: string) = + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok _ -> Assert.Pass() + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index ec6dc96..08c95f9 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -15,7 +15,7 @@ type IrArtifactTests() = [] member _.``IR artifact is deterministic and executable after round trip`` () = - let compiled = compile "let add x y = x + y\nadd 1 2" + let compiled = compile "let add x y = x + y\n(Some (add 1 2), [1] @ [2])" let first = IrArtifact.serialize compiled.Lir let second = IrArtifact.serialize compiled.Lir Assert.That((first = second), Is.True) @@ -23,7 +23,7 @@ type IrArtifactTests() = | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) | Ok loaded -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } loaded with - | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 3L)) + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VTuple [ LirValue.VOption(Some(LirValue.VInt 3L)); LirValue.VList [ LirValue.VInt 1L; LirValue.VInt 2L ] ])) | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) [] From a3d9d8a25c161f5fc317ce578bb6cb74fa307d72 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 16:47:13 +0200 Subject: [PATCH 009/133] feat(compiler): lower lexical bindings and failures --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 ++ src/FScript.Compiler/GoBackend.fs | 15 ++++++++++++- src/FScript.Compiler/Hir.fs | 7 +++++++ src/FScript.Compiler/HirLowering.fs | 10 +++++++++ src/FScript.Compiler/HirValidation.fs | 6 ++++++ src/FScript.Compiler/IrArtifact.fs | 8 +++++++ src/FScript.Compiler/Lir.fs | 4 ++++ src/FScript.Compiler/LirInterpreter.fs | 16 ++++++++++++++ .../FScript.Compiler.Tests/GoBackendTests.fs | 2 ++ tests/FScript.Compiler.Tests/HirTests.fs | 21 +++++++++++++++++++ .../FScript.Compiler.Tests/IrArtifactTests.fs | 8 +++++++ 12 files changed, 99 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1fe75ae..b5fa767 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added compiled local bindings, recursive lexical closures, and explicit runtime failures to HIR, LIR, and Go projects. - Added compiler, serialized LIR, interpreter, and Go support for tuples, records, options, lists, append, and recursive structural equality. - Added `fscript compile` targets for whole-program `.fsir` artifacts and deterministic Go source projects. - Added deterministic, versioned JSON `.fsir` serialization and validated deserialization for executable LIR. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 6c18cd4..012a6ee 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -20,6 +20,8 @@ HIR is language-oriented and contains resolved artifact-local symbol identities, LIR is executable but not a Go AST. It uses lifted functions, explicit captures and closure creation, resolved primitive operations, aggregate construction/projection, and structured control flow. Pattern matching becomes deterministic tests and projections before reaching a backend. Semantic types may remain alongside selected runtime representations for validation and diagnostics. +Lexical `let` is retained as a structured LIR node. Its symbol is visible only in the body, or in both the binding and body for recursive functions. The interpreter closes the recursive environment explicitly; Go emission declares the function slot before assigning its closure. `raise` becomes an explicit `Fail` node so consumers preserve evaluation order and attach their own stage-specific failure diagnostic. + Serialized `.fsir` files contain one closed resolved program plus explicit foreign linkage declarations. JSON format versions are internal and may be rejected rather than upgraded. ## Representation policy diff --git a/src/FScript.Compiler/GoBackend.fs b/src/FScript.Compiler/GoBackend.fs index 1652597..01512ce 100644 --- a/src/FScript.Compiler/GoBackend.fs +++ b/src/FScript.Compiler/GoBackend.fs @@ -101,6 +101,19 @@ module GoBackend = match emitExpression environment left, emitExpression environment right with | Ok l, Ok r -> Ok $"fsruntime.Append({l}, {r})" | Error error, _ | _, Error error -> Error error + | Lir.Let(id, _, recursive, binding, body) -> + let name = identifier "local" id + let scoped = environment |> Map.add id name + match emitExpression scoped binding, emitExpression scoped body with + | Ok bindingCode, Ok bodyCode -> + let declaration = + if recursive then $"var {name} {goType binding.Type}; {name} = {bindingCode}" + else $"{name} := {bindingCode}" + Ok $"func() {goType expression.Type} {{ {declaration}; return {bodyCode} }}()" + | Error error, _ | _, Error error -> Error error + | Lir.Fail value -> + emitExpression environment value + |> Result.map (fun code -> $"func() {goType expression.Type} {{ fsruntime.Fail({code}); var zero {goType expression.Type}; return zero }}()") | Lir.Operation _ -> Error "Invalid operation arity" and private collect<'a> (results: Result<'a, string> list) : Result<'a list, string> = @@ -134,6 +147,6 @@ module GoBackend = [ { RelativePath = "go.mod"; Content = $"module {moduleName}\n\ngo 1.25\n" } { RelativePath = "main.go"; Content = $"package main\n\nimport (\n\t\"fmt\"\n\t\"{moduleName}/internal/fscriptprogram\"\n)\n\nfunc main() {{ fmt.Println(fscriptprogram.Run()) }}\n" } { RelativePath = "internal/fscriptprogram/program.go"; Content = programSource } - { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nimport \"reflect\"\n\ntype Option struct { HasValue bool; Value any }\n\nfunc Int(value any) int64 { return value.(int64) }\nfunc Some(value any) Option { return Option{HasValue: true, Value: value} }\nfunc None() Option { return Option{} }\nfunc Equal(left any, right any) bool { return reflect.DeepEqual(left, right) }\nfunc Cons(head any, tail []any) []any { return append([]any{head}, tail...) }\nfunc Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) }\n" } ] + { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n)\n\ntype Option struct { HasValue bool; Value any }\n\nfunc Int(value any) int64 { return value.(int64) }\nfunc Some(value any) Option { return Option{HasValue: true, Value: value} }\nfunc None() Option { return Option{} }\nfunc Equal(left any, right any) bool { return reflect.DeepEqual(left, right) }\nfunc Cons(head any, tail []any) []any { return append([]any{head}, tail...) }\nfunc Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) }\nfunc Fail(value any) { panic(fmt.Sprint(value)) }\n" } ] with ex -> Error [ { Code = "FSB4002"; Stage = GoGeneration; Message = ex.Message; Span = None } ] diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs index a0b4fac..aed4416 100644 --- a/src/FScript.Compiler/Hir.fs +++ b/src/FScript.Compiler/Hir.fs @@ -50,6 +50,8 @@ module Hir = | NoneValue | Cons of Expression * Expression | Append of Expression * Expression + | Let of SymbolId * string * bool * Expression * Expression + | Fail of Expression type Declaration = { Id: SymbolId @@ -112,6 +114,11 @@ module Hir = | NoneValue -> lines.Add(typed "none") | Cons(head, tail) -> lines.Add(typed "cons"); expression (indent + 2) head; expression (indent + 2) tail | Append(left, right) -> lines.Add(typed "append"); expression (indent + 2) left; expression (indent + 2) right + | Let(id, name, recursive, binding, body) -> + lines.Add(typed (sprintf "let%s %%%d %s" (if recursive then " rec" else "") (symbolNumber id) name)) + expression (indent + 2) binding + expression (indent + 2) body + | Fail value -> lines.Add(typed "fail"); expression (indent + 2) value for declaration in program.Declarations do lines.Add(sprintf "decl %%%d %s: %s" (symbolNumber declaration.Id) declaration.Name (formatType (let (Forall(_, value)) = declaration.Scheme in value))) diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index fe9b225..ea2aca7 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -135,6 +135,16 @@ module HirLowering = | EAppend(left, right, _) -> match lowerExpression state environment inferred left, lowerExpression state environment inferred right with | Ok l, Ok r -> Ok(make (Hir.Append(l, r))) | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e + | ELet(name, binding, body, isRecursive, _, _) -> + let id = newSymbol state + let bindingType = expressionType state binding (Types.freshVar()) + let bodyEnvironment = environment |> Map.add name id + let bindingEnvironment = if isRecursive then bodyEnvironment else environment + match lowerExpression state bindingEnvironment bindingType binding, lowerExpression state bodyEnvironment inferred body with + | Ok value, Ok loweredBody -> Ok(make (Hir.Let(id, name, isRecursive, value, loweredBody))) + | Error a, Error b -> Error(a @ b) + | Error errors, _ | _, Error errors -> Error errors + | ERaise(value, _) -> lowerExpression state environment TString value |> Result.map (Hir.Fail >> make) | _ -> Error [ Diagnostic.hir "FSH1099" $"HIR lowering does not yet support {expression.GetType().Name}." (Some span) ] and private collectResults<'a> (results: Result<'a, CompilerDiagnostic list> list) : Result<'a list, CompilerDiagnostic list> = diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index 706e160..f0d3bdd 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -21,6 +21,12 @@ module HirValidation = | Hir.FieldGet(target, _) | Hir.SomeValue target -> references target | Hir.NoneValue -> Set.empty | Hir.Cons(head, tail) | Hir.Append(head, tail) -> Set.union (references head) (references tail) + | Hir.Let(id, _, recursive, binding, body) -> + let bindingReferences = + let value = references binding + if recursive then Set.remove id value else value + Set.union bindingReferences (references body |> Set.remove id) + | Hir.Fail value -> references value let declared = declarationIds |> Set.ofList let unresolved = [ yield! program.Declarations |> List.collect (fun value -> references value.Value |> Set.toList) diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index ebf2cc6..f939c6d 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -109,6 +109,11 @@ module IrArtifact = | Lir.Append(left, right) -> writer.WriteString("kind", "append"); writer.WritePropertyName("left"); writeExpression writer left writer.WritePropertyName("right"); writeExpression writer right + | Lir.Let(id, name, recursive, binding, body) -> + writer.WriteString("kind", "let"); writer.WriteNumber("symbol", Lir.symbolNumber id); writer.WriteString("name", name) + writer.WriteBoolean("recursive", recursive); writer.WritePropertyName("binding"); writeExpression writer binding + writer.WritePropertyName("body"); writeExpression writer body + | Lir.Fail value -> writer.WriteString("kind", "fail"); writer.WritePropertyName("value"); writeExpression writer value writer.WriteEndObject() let serialize (program: Lir.Program) = @@ -188,6 +193,9 @@ module IrArtifact = | "none" -> Lir.NoneValue | "cons" -> Lir.Cons(readExpression (property "head" element), readExpression (property "tail" element)) | "append" -> Lir.Append(readExpression (property "left" element), readExpression (property "right" element)) + | "let" -> Lir.Let(Lir.SymbolId((property "symbol" element).GetInt32()), stringProperty "name" element, + (property "recursive" element).GetBoolean(), readExpression (property "binding" element), readExpression (property "body" element)) + | "fail" -> Lir.Fail(readExpression (property "value" element)) | kind -> invalidOp $"Unknown LIR expression kind '{kind}'." { Type = expressionType; Span = span; Node = node } diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index 7221845..c57d5b9 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -38,6 +38,8 @@ module Lir = | NoneValue | Cons of Expression * Expression | Append of Expression * Expression + | Let of SymbolId * string * bool * Expression * Expression + | Fail of Expression type Declaration = { Id: SymbolId @@ -89,6 +91,8 @@ module LirLowering = | Hir.NoneValue -> Lir.NoneValue | Hir.Cons(head, tail) -> Lir.Cons(expression head, expression tail) | Hir.Append(left, right) -> Lir.Append(expression left, expression right) + | Hir.Let(id, name, recursive, binding, body) -> Lir.Let(symbol id, name, recursive, expression binding, expression body) + | Hir.Fail value -> Lir.Fail(expression value) { Type = value.Type; Span = value.Span; Node = node } let lower (program: Hir.Program) : Lir.Program = diff --git a/src/FScript.Compiler/LirInterpreter.fs b/src/FScript.Compiler/LirInterpreter.fs index bf2027f..57673cb 100644 --- a/src/FScript.Compiler/LirInterpreter.fs +++ b/src/FScript.Compiler/LirInterpreter.fs @@ -107,6 +107,22 @@ module LirInterpreter = match evaluate environment left, evaluate environment right with | Ok(VList l), Ok(VList r) -> Ok(VList(l @ r)) | Ok _, Ok _ -> fail "FSI3009" expression.Span "Append operands must be lists." | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e + | Lir.Let(id, _, recursive, binding, body) -> + if recursive then + match evaluate environment binding with + | Ok(VClosure(_, _, _, captured) as closure) -> + let next = environment |> Map.add id closure + captured.Value <- next + evaluate next body + | Ok _ -> fail "FSI3010" expression.Span "Recursive local bindings must be functions." + | Error errors -> Error errors + else + evaluate environment binding |> Result.bind (fun value -> evaluate (environment |> Map.add id value) body) + | Lir.Fail value -> + match evaluate environment value with + | Ok(VString message) -> fail "FSI3011" expression.Span message + | Ok _ -> fail "FSI3011" expression.Span "FScript program raised a non-string value." + | Error errors -> Error errors and private evaluateMany environment items = items |> List.map (evaluate environment) |> collect diff --git a/tests/FScript.Compiler.Tests/GoBackendTests.fs b/tests/FScript.Compiler.Tests/GoBackendTests.fs index e7e8073..fea128d 100644 --- a/tests/FScript.Compiler.Tests/GoBackendTests.fs +++ b/tests/FScript.Compiler.Tests/GoBackendTests.fs @@ -12,6 +12,8 @@ type GoBackendTests() = [] [] [] + [] + [] member _.``Generated projects run with Go`` (source: string, expected: string) = let input = { Program = Parser.parseProgram source; Externs = [] } let compiled = diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index 3023938..60d789e 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -42,3 +42,24 @@ type HirTests() = match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok _ -> Assert.Pass() | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``Executes lexical let bindings and recursive closures`` () = + let source = "let f x = (let rec loop n = if n = 0 then x else loop (n - 1)\n loop 2\n)\nf 7" + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 7L)) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``Raise preserves an interpreter diagnostic`` () = + match Compiler.compile { Program = Parser.parseProgram "raise \"boom\""; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok _ -> Assert.Fail("Expected failure") + | Error diagnostics -> + Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSI3011")) + Assert.That(diagnostics |> List.map _.Message, Does.Contain("boom")) diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index 08c95f9..2dec03e 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -39,3 +39,11 @@ type IrArtifactTests() = match IrArtifact.deserialize (Encoding.UTF8.GetBytes("{")) with | Ok _ -> Assert.Fail("Expected corrupt-artifact diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSA5003")) + + [] + member _.``IR artifact round trips lexical let and fail nodes`` () = + for source in [ "let f x = (let y = x + 1\n y\n)\nf 2"; "raise \"failure\"" ] do + let bytes = compile source |> _.Lir |> IrArtifact.serialize + match IrArtifact.deserialize bytes with + | Ok _ -> () + | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) From 461d50e96e949216794988d50271e0f114a981a8 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 16:52:41 +0200 Subject: [PATCH 010/133] feat(compiler): lower pattern decision trees --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 + src/FScript.Compiler/GoBackend.fs | 70 +++++++++++++++- src/FScript.Compiler/Hir.fs | 29 ++++++- src/FScript.Compiler/HirLowering.fs | 63 +++++++++++++-- src/FScript.Compiler/HirValidation.fs | 14 ++++ src/FScript.Compiler/IrArtifact.fs | 80 ++++++++++++++++--- src/FScript.Compiler/Lir.fs | 43 ++++++++++ src/FScript.Compiler/LirInterpreter.fs | 46 ++++++++++- .../FScript.Compiler.Tests/GoBackendTests.fs | 2 + tests/FScript.Compiler.Tests/HirTests.fs | 12 +++ .../FScript.Compiler.Tests/IrArtifactTests.fs | 2 +- 12 files changed, 342 insertions(+), 22 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b5fa767..b86ff72 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added deterministic LIR pattern decision trees with single scrutinee evaluation, ordered guards, bindings, and match-failure diagnostics. - Added compiled local bindings, recursive lexical closures, and explicit runtime failures to HIR, LIR, and Go projects. - Added compiler, serialized LIR, interpreter, and Go support for tuples, records, options, lists, append, and recursive structural equality. - Added `fscript compile` targets for whole-program `.fsir` artifacts and deterministic Go source projects. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 012a6ee..ce15061 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -22,6 +22,8 @@ LIR is executable but not a Go AST. It uses lifted functions, explicit captures Lexical `let` is retained as a structured LIR node. Its symbol is visible only in the body, or in both the binding and body for recursive functions. The interpreter closes the recursive environment explicitly; Go emission declares the function slot before assigning its closure. `raise` becomes an explicit `Fail` node so consumers preserve evaluation order and attach their own stage-specific failure diagnostic. +Matches lower to a structured decision tree whose target is evaluated once. Cases remain ordered; each pattern test creates artifact-local bindings, then its guard is evaluated, and only then is its body selected. A failed guard continues at the next case. Exhaustion is the explicit FScript match-failure path. Literal, list, tuple, record, and option patterns use the same tree in serialized artifacts, the LIR interpreter, and generated Go. + Serialized `.fsir` files contain one closed resolved program plus explicit foreign linkage declarations. JSON format versions are internal and may be rejected rather than upgraded. ## Representation policy diff --git a/src/FScript.Compiler/GoBackend.fs b/src/FScript.Compiler/GoBackend.fs index 01512ce..696e177 100644 --- a/src/FScript.Compiler/GoBackend.fs +++ b/src/FScript.Compiler/GoBackend.fs @@ -46,6 +46,57 @@ module GoBackend = | Lir.BooleanAnd -> Some "&&" | Lir.BooleanOr -> Some "||" + let private cast valueType code = + match valueType with + | TInt -> $"fsruntime.Int({code})" + | TFloat -> $"fsruntime.Float({code})" + | TBool -> $"fsruntime.Bool({code})" + | TString -> $"fsruntime.String({code})" + | TList _ | TTuple _ -> $"fsruntime.List({code})" + | TRecord _ -> $"fsruntime.Record({code})" + | TOption _ -> $"fsruntime.AsOption({code})" + | _ -> code + + let rec private patternBindings (pattern: Lir.Pattern) = + match pattern.Node with + | Lir.Binding(id, _) -> [ id ] + | Lir.ConsPattern(head, tail) -> patternBindings head @ patternBindings tail + | Lir.TuplePattern items -> List.collect patternBindings items + | Lir.RecordPattern fields -> fields |> List.collect (snd >> patternBindings) + | Lir.SomePattern item -> patternBindings item + | _ -> [] + + let rec private emitPattern valueCode (pattern: Lir.Pattern) = + let combine parts = + let tests, assignments = List.unzip parts + let test = tests |> List.filter ((<>) "true") |> function [] -> "true" | values -> String.concat " && " values + test, List.concat assignments + match pattern.Node with + | Lir.Wildcard -> "true", [] + | Lir.Binding(id, _) -> + let name = identifier "match" id + "true", [ $"{name} := {cast pattern.Type valueCode}; _ = {name}" ] + | Lir.LiteralPattern value -> $"fsruntime.Equal({valueCode}, {constant value})", [] + | Lir.EmptyList -> $"len(fsruntime.List({valueCode})) == 0", [] + | Lir.NonePattern -> $"!fsruntime.IsSome({valueCode})", [] + | Lir.SomePattern item -> + let test, assignments = emitPattern $"fsruntime.OptionValue({valueCode})" item + $"fsruntime.IsSome({valueCode}) && ({test})", assignments + | Lir.ConsPattern(head, tail) -> + let headPart = emitPattern $"fsruntime.List({valueCode})[0]" head + let tailPart = emitPattern $"fsruntime.Tail({valueCode})" tail + let test, assignments = combine [ headPart; tailPart ] + $"len(fsruntime.List({valueCode})) > 0 && ({test})", assignments + | Lir.TuplePattern items -> + items |> List.mapi (fun index item -> emitPattern $"fsruntime.List({valueCode})[{index}]" item) + |> combine |> fun (test, assignments) -> $"len(fsruntime.List({valueCode})) == {items.Length} && ({test})", assignments + | Lir.RecordPattern fields -> + fields |> List.map (fun (name, item) -> + let quoted = System.Text.Json.JsonSerializer.Serialize(name) + let test, assignments = emitPattern $"fsruntime.Record({valueCode})[{quoted}]" item + $"fsruntime.HasField({valueCode}, {quoted}) && ({test})", assignments) + |> combine + let rec private emitExpression environment (expression: Lir.Expression) = match expression.Node with | Lir.Constant value -> Ok(constant value) @@ -114,6 +165,23 @@ module GoBackend = | Lir.Fail value -> emitExpression environment value |> Result.map (fun code -> $"func() {goType expression.Type} {{ fsruntime.Fail({code}); var zero {goType expression.Type}; return zero }}()") + | Lir.DecisionTree(target, cases) -> + emitExpression environment target |> Result.bind (fun targetCode -> + let mutable failure = None + let clauses = StringBuilder() + for case in cases do + let bindings = patternBindings case.Pattern + let caseEnvironment = bindings |> List.fold (fun env id -> Map.add id (identifier "match" id) env) environment + match case.Guard |> Option.map (emitExpression caseEnvironment) |> Option.defaultValue (Ok "true"), emitExpression caseEnvironment case.Body with + | Ok guardCode, Ok bodyCode -> + let test, assignments = emitPattern "matchValue" case.Pattern + clauses.Append($"if {test} {{ ") |> ignore + assignments |> List.iter (fun assignment -> clauses.Append(assignment).Append("; ") |> ignore) + clauses.Append($"if {guardCode} {{ return {bodyCode} }} }}; ") |> ignore + | Error error, _ | _, Error error -> failure <- Some error + match failure with + | Some error -> Error error + | None -> Ok $"func() {goType expression.Type} {{ matchValue := {targetCode}; {clauses}fsruntime.Fail(\"Match failure.\"); var zero {goType expression.Type}; return zero }}()") | Lir.Operation _ -> Error "Invalid operation arity" and private collect<'a> (results: Result<'a, string> list) : Result<'a list, string> = @@ -147,6 +215,6 @@ module GoBackend = [ { RelativePath = "go.mod"; Content = $"module {moduleName}\n\ngo 1.25\n" } { RelativePath = "main.go"; Content = $"package main\n\nimport (\n\t\"fmt\"\n\t\"{moduleName}/internal/fscriptprogram\"\n)\n\nfunc main() {{ fmt.Println(fscriptprogram.Run()) }}\n" } { RelativePath = "internal/fscriptprogram/program.go"; Content = programSource } - { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n)\n\ntype Option struct { HasValue bool; Value any }\n\nfunc Int(value any) int64 { return value.(int64) }\nfunc Some(value any) Option { return Option{HasValue: true, Value: value} }\nfunc None() Option { return Option{} }\nfunc Equal(left any, right any) bool { return reflect.DeepEqual(left, right) }\nfunc Cons(head any, tail []any) []any { return append([]any{head}, tail...) }\nfunc Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) }\nfunc Fail(value any) { panic(fmt.Sprint(value)) }\n" } ] + { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n)\n\ntype Option struct { HasValue bool; Value any }\n\nfunc Int(value any) int64 { return value.(int64) }\nfunc Float(value any) float64 { return value.(float64) }\nfunc Bool(value any) bool { return value.(bool) }\nfunc String(value any) string { return value.(string) }\nfunc List(value any) []any { return value.([]any) }\nfunc Record(value any) map[string]any { return value.(map[string]any) }\nfunc AsOption(value any) Option { return value.(Option) }\nfunc Some(value any) Option { return Option{HasValue: true, Value: value} }\nfunc None() Option { return Option{} }\nfunc IsSome(value any) bool { return AsOption(value).HasValue }\nfunc OptionValue(value any) any { return AsOption(value).Value }\nfunc HasField(value any, name string) bool { _, ok := Record(value)[name]; return ok }\nfunc Equal(left any, right any) bool { return reflect.DeepEqual(left, right) }\nfunc Cons(head any, tail []any) []any { return append([]any{head}, tail...) }\nfunc Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) }\nfunc Tail(value any) []any { return append([]any{}, List(value)[1:]...) }\nfunc Fail(value any) { panic(fmt.Sprint(value)) }\n" } ] with ex -> Error [ { Code = "FSB4002"; Stage = GoGeneration; Message = ex.Message; Span = None } ] diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs index aed4416..b079e43 100644 --- a/src/FScript.Compiler/Hir.fs +++ b/src/FScript.Compiler/Hir.fs @@ -52,6 +52,29 @@ module Hir = | Append of Expression * Expression | Let of SymbolId * string * bool * Expression * Expression | Fail of Expression + | Match of Expression * MatchCase list + + and Pattern = + { Type: Type + Span: Span + Node: PatternNode } + + and PatternNode = + | Wildcard + | Binding of SymbolId * string + | LiteralPattern of LiteralValue + | EmptyList + | ConsPattern of Pattern * Pattern + | TuplePattern of Pattern list + | RecordPattern of (string * Pattern) list + | SomePattern of Pattern + | NonePattern + + and MatchCase = + { Pattern: Pattern + Guard: Expression option + Body: Expression + Span: Span } type Declaration = { Id: SymbolId @@ -82,7 +105,7 @@ module Hir = let format (program: Program) = let lines = ResizeArray() - let rec expression indent value = + let rec expression indent (value: Expression) = let prefix = System.String(' ', indent) let typed text = $"{prefix}{text} : {formatType value.Type}" match value.Node with @@ -119,6 +142,10 @@ module Hir = expression (indent + 2) binding expression (indent + 2) body | Fail value -> lines.Add(typed "fail"); expression (indent + 2) value + | Match(target, cases) -> + lines.Add(typed $"match {cases.Length} cases") + expression (indent + 2) target + cases |> List.iter (fun case -> expression (indent + 2) case.Body) for declaration in program.Declarations do lines.Add(sprintf "decl %%%d %s: %s" (symbolNumber declaration.Id) declaration.Name (formatType (let (Forall(_, value)) = declaration.Scheme in value))) diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index ea2aca7..0d219f7 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -42,6 +42,49 @@ module HirLowering = | "||", TBool -> Some Hir.BooleanOr | _ -> None + let private collectResults<'a> (results: Result<'a, CompilerDiagnostic list> list) : Result<'a list, CompilerDiagnostic list> = + let folder state result = + match state, result with + | Ok values, Ok value -> Ok(values @ [ value ]) + | Error a, Error b -> Error(a @ b) + | Error errors, _ | _, Error errors -> Error errors + results |> List.fold folder (Ok []) + + let rec private lowerPattern state expected pattern = + let span = Ast.spanOfPattern pattern + let make node = ({ Type = expected; Span = span; Node = node }: Hir.Pattern) + match pattern with + | PWildcard _ -> Ok(make Hir.Wildcard, Map.empty) + | PVar(name, _) -> + let id = newSymbol state + Ok(make (Hir.Binding(id, name)), Map.ofList [ name, id ]) + | PLiteral(value, _) -> let lowered, _ = literal value in Ok(make (Hir.LiteralPattern lowered), Map.empty) + | PNil _ -> Ok(make Hir.EmptyList, Map.empty) + | PNone _ -> Ok(make Hir.NonePattern, Map.empty) + | PSome(inner, _) -> + let itemType = match expected with TOption value -> value | _ -> Types.freshVar() + lowerPattern state itemType inner |> Result.map (fun (value, bindings) -> make (Hir.SomePattern value), bindings) + | PCons(head, tail, _) -> + let itemType = match expected with TList value -> value | _ -> Types.freshVar() + match lowerPattern state itemType head, lowerPattern state expected tail with + | Ok(h, hb), Ok(t, tb) -> Ok(make (Hir.ConsPattern(h, t)), Map.fold (fun env key value -> Map.add key value env) hb tb) + | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e + | PTuple(items, _) -> + let types = match expected with TTuple values when values.Length = items.Length -> values | _ -> List.replicate items.Length (Types.freshVar()) + List.zip items types |> List.map (fun (item, itemType) -> lowerPattern state itemType item) + |> collectResults |> Result.map (fun values -> + let patterns, bindingMaps = List.unzip values + let bindings = bindingMaps |> List.fold (Map.fold (fun env key value -> Map.add key value env)) Map.empty + make (Hir.TuplePattern patterns), bindings) + | PRecord(fields, _) -> + let types = match expected with TRecord values -> values | _ -> Map.empty + fields |> List.map (fun (name, item) -> lowerPattern state (types |> Map.tryFind name |> Option.defaultValue (Types.freshVar())) item |> Result.map (fun (value, bindings) -> (name, value), bindings)) + |> collectResults |> Result.map (fun values -> + let fields, bindingMaps = List.unzip values + let bindings = bindingMaps |> List.fold (Map.fold (fun env key value -> Map.add key value env)) Map.empty + make (Hir.RecordPattern fields), bindings) + | _ -> Error [ Diagnostic.hir "FSH1004" "Pattern form is not yet supported by HIR lowering." (Some span) ] + let rec private lowerExpression state environment fallback expression = let span = Ast.spanOfExpr expression let inferred = expressionType state expression fallback @@ -145,16 +188,20 @@ module HirLowering = | Error a, Error b -> Error(a @ b) | Error errors, _ | _, Error errors -> Error errors | ERaise(value, _) -> lowerExpression state environment TString value |> Result.map (Hir.Fail >> make) + | EMatch(target, cases, _) -> + let targetType = expressionType state target (Types.freshVar()) + match lowerExpression state environment targetType target with + | Error errors -> Error errors + | Ok loweredTarget -> + cases |> List.map (fun (pattern, guard, body, caseSpan) -> + lowerPattern state targetType pattern |> Result.bind (fun (loweredPattern, bindings) -> + let caseEnvironment = Map.fold (fun env name id -> Map.add name id env) environment bindings + match guard |> Option.map (fun value -> lowerExpression state caseEnvironment TBool value |> Result.map Some) |> Option.defaultValue (Ok None), lowerExpression state caseEnvironment inferred body with + | Ok loweredGuard, Ok loweredBody -> Ok({ Pattern = loweredPattern; Guard = loweredGuard; Body = loweredBody; Span = caseSpan }: Hir.MatchCase) + | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e)) + |> collectResults |> Result.map (fun loweredCases -> make (Hir.Match(loweredTarget, loweredCases))) | _ -> Error [ Diagnostic.hir "FSH1099" $"HIR lowering does not yet support {expression.GetType().Name}." (Some span) ] - and private collectResults<'a> (results: Result<'a, CompilerDiagnostic list> list) : Result<'a list, CompilerDiagnostic list> = - let folder state result = - match state, result with - | Ok values, Ok value -> Ok(values @ [ value ]) - | Error a, Error b -> Error(a @ b) - | Error errors, _ | _, Error errors -> Error errors - results |> List.fold folder (Ok []) - let lower (resolved: TypeInfer.ResolvedProgram) : Result = let expressionTypes = resolved.ExpressionTypes diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index f0d3bdd..280e463 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -27,6 +27,20 @@ module HirValidation = if recursive then Set.remove id value else value Set.union bindingReferences (references body |> Set.remove id) | Hir.Fail value -> references value + | Hir.Match(target, cases) -> + let rec bindings (pattern: Hir.Pattern) = + match pattern.Node with + | Hir.Binding(id, _) -> Set.singleton id + | Hir.ConsPattern(head, tail) -> Set.union (bindings head) (bindings tail) + | Hir.TuplePattern items -> items |> List.map bindings |> Set.unionMany + | Hir.RecordPattern fields -> fields |> List.map (snd >> bindings) |> Set.unionMany + | Hir.SomePattern item -> bindings item + | _ -> Set.empty + let caseReferences (case: Hir.MatchCase) = + let bound = bindings case.Pattern + let values = Set.union (case.Guard |> Option.map references |> Option.defaultValue Set.empty) (references case.Body) + Set.difference values bound + Set.union (references target) (cases |> List.map caseReferences |> Set.unionMany) let declared = declarationIds |> Set.ofList let unresolved = [ yield! program.Declarations |> List.collect (fun value -> references value.Value |> Set.toList) diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index f939c6d..f2306d2 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -65,15 +65,40 @@ module IrArtifact = let private constantName = function | Lir.Unit -> "unit" | Lir.Int _ -> "int" | Lir.Float _ -> "float" | Lir.Bool _ -> "bool" | Lir.String _ -> "string" + let private writeConstantValue (writer: Utf8JsonWriter) constant = + writer.WriteString("constantKind", constantName constant) + match constant with + | Lir.Unit -> () | Lir.Int value -> writer.WriteNumber("intValue", value) | Lir.Float value -> writer.WriteNumber("floatValue", value) + | Lir.Bool value -> writer.WriteBoolean("boolValue", value) | Lir.String value -> writer.WriteString("stringValue", value) + + let rec private writePattern (writer: Utf8JsonWriter) (value: Lir.Pattern) = + writer.WriteStartObject(); writer.WritePropertyName("type"); writeType writer value.Type + writer.WritePropertyName("span"); writeSpan writer value.Span + match value.Node with + | Lir.Wildcard -> writer.WriteString("kind", "wildcard") + | Lir.Binding(id, name) -> writer.WriteString("kind", "binding"); writer.WriteNumber("symbol", Lir.symbolNumber id); writer.WriteString("name", name) + | Lir.LiteralPattern constant -> writer.WriteString("kind", "literal"); writeConstantValue writer constant + | Lir.EmptyList -> writer.WriteString("kind", "empty-list") + | Lir.NonePattern -> writer.WriteString("kind", "none") + | Lir.SomePattern item -> writer.WriteString("kind", "some"); writer.WritePropertyName("item"); writePattern writer item + | Lir.ConsPattern(head, tail) -> + writer.WriteString("kind", "cons"); writer.WritePropertyName("head"); writePattern writer head + writer.WritePropertyName("tail"); writePattern writer tail + | Lir.TuplePattern items -> + writer.WriteString("kind", "tuple"); writer.WriteStartArray("items"); items |> List.iter (writePattern writer); writer.WriteEndArray() + | Lir.RecordPattern fields -> + writer.WriteString("kind", "record"); writer.WriteStartArray("fields") + for name, item in fields do + writer.WriteStartObject(); writer.WriteString("name", name); writer.WritePropertyName("pattern"); writePattern writer item; writer.WriteEndObject() + writer.WriteEndArray() + writer.WriteEndObject() + let rec private writeExpression (writer: Utf8JsonWriter) (value: Lir.Expression) = writer.WriteStartObject(); writer.WritePropertyName("type"); writeType writer value.Type writer.WritePropertyName("span"); writeSpan writer value.Span match value.Node with | Lir.Constant constant -> - writer.WriteString("kind", "constant"); writer.WriteString("constantKind", constantName constant) - match constant with - | Lir.Unit -> () | Lir.Int v -> writer.WriteNumber("intValue", v) | Lir.Float v -> writer.WriteNumber("floatValue", v) - | Lir.Bool v -> writer.WriteBoolean("boolValue", v) | Lir.String v -> writer.WriteString("stringValue", v) + writer.WriteString("kind", "constant"); writeConstantValue writer constant | Lir.Load id -> writer.WriteString("kind", "load"); writer.WriteNumber("symbol", Lir.symbolNumber id) | Lir.MakeClosure(id, name, parameterType, body) -> writer.WriteString("kind", "closure"); writer.WriteNumber("symbol", Lir.symbolNumber id); writer.WriteString("name", name) @@ -114,6 +139,15 @@ module IrArtifact = writer.WriteBoolean("recursive", recursive); writer.WritePropertyName("binding"); writeExpression writer binding writer.WritePropertyName("body"); writeExpression writer body | Lir.Fail value -> writer.WriteString("kind", "fail"); writer.WritePropertyName("value"); writeExpression writer value + | Lir.DecisionTree(target, cases) -> + writer.WriteString("kind", "decision-tree"); writer.WritePropertyName("target"); writeExpression writer target + writer.WriteStartArray("cases") + for case in cases do + writer.WriteStartObject(); writer.WritePropertyName("pattern"); writePattern writer case.Pattern + match case.Guard with Some guard -> writer.WritePropertyName("guard"); writeExpression writer guard | None -> writer.WriteNull("guard") + writer.WritePropertyName("body"); writeExpression writer case.Body; writer.WritePropertyName("span"); writeSpan writer case.Span + writer.WriteEndObject() + writer.WriteEndArray() writer.WriteEndObject() let serialize (program: Lir.Program) = @@ -167,17 +201,35 @@ module IrArtifact = | "NumericGreaterThanOrEqual" -> Lir.NumericGreaterThanOrEqual | "BooleanAnd" -> Lir.BooleanAnd | "BooleanOr" -> Lir.BooleanOr | value -> invalidOp $"Unknown LIR operation '{value}'." + let private readConstant (element: JsonElement) = + match stringProperty "constantKind" element with + | "unit" -> Lir.Unit | "int" -> Lir.Int((property "intValue" element).GetInt64()) + | "float" -> Lir.Float((property "floatValue" element).GetDouble()) + | "bool" -> Lir.Bool((property "boolValue" element).GetBoolean()) + | "string" -> Lir.String(stringProperty "stringValue" element) + | value -> invalidOp $"Unknown constant '{value}'." + + let rec private readPattern (element: JsonElement) : Lir.Pattern = + let node = + match stringProperty "kind" element with + | "wildcard" -> Lir.Wildcard + | "binding" -> Lir.Binding(Lir.SymbolId((property "symbol" element).GetInt32()), stringProperty "name" element) + | "literal" -> Lir.LiteralPattern(readConstant element) + | "empty-list" -> Lir.EmptyList + | "none" -> Lir.NonePattern + | "some" -> Lir.SomePattern(readPattern (property "item" element)) + | "cons" -> Lir.ConsPattern(readPattern (property "head" element), readPattern (property "tail" element)) + | "tuple" -> Lir.TuplePattern((property "items" element).EnumerateArray() |> Seq.map readPattern |> Seq.toList) + | "record" -> Lir.RecordPattern((property "fields" element).EnumerateArray() |> Seq.map (fun field -> stringProperty "name" field, readPattern (property "pattern" field)) |> Seq.toList) + | kind -> invalidOp $"Unknown LIR pattern kind '{kind}'." + { Type = readType (property "type" element); Span = readSpan (property "span" element); Node = node } + let rec private readExpression (element: JsonElement) : Lir.Expression = let expressionType = readType (property "type" element) let span = readSpan (property "span" element) let node = match stringProperty "kind" element with - | "constant" -> - Lir.Constant(match stringProperty "constantKind" element with - | "unit" -> Lir.Unit | "int" -> Lir.Int((property "intValue" element).GetInt64()) - | "float" -> Lir.Float((property "floatValue" element).GetDouble()) - | "bool" -> Lir.Bool((property "boolValue" element).GetBoolean()) - | "string" -> Lir.String(stringProperty "stringValue" element) | value -> invalidOp $"Unknown constant '{value}'.") + | "constant" -> Lir.Constant(readConstant element) | "load" -> Lir.Load(Lir.SymbolId((property "symbol" element).GetInt32())) | "closure" -> Lir.MakeClosure(Lir.SymbolId((property "symbol" element).GetInt32()), stringProperty "name" element, readType (property "parameterType" element), readExpression (property "body" element)) @@ -196,6 +248,14 @@ module IrArtifact = | "let" -> Lir.Let(Lir.SymbolId((property "symbol" element).GetInt32()), stringProperty "name" element, (property "recursive" element).GetBoolean(), readExpression (property "binding" element), readExpression (property "body" element)) | "fail" -> Lir.Fail(readExpression (property "value" element)) + | "decision-tree" -> + let cases = (property "cases" element).EnumerateArray() |> Seq.map (fun item -> + let guard = property "guard" item + ({ Pattern = readPattern (property "pattern" item) + Guard = if guard.ValueKind = JsonValueKind.Null then None else Some(readExpression guard) + Body = readExpression (property "body" item) + Span = readSpan (property "span" item) }: Lir.DecisionCase)) |> Seq.toList + Lir.DecisionTree(readExpression (property "target" element), cases) | kind -> invalidOp $"Unknown LIR expression kind '{kind}'." { Type = expressionType; Span = span; Node = node } diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index c57d5b9..9a9ec85 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -40,6 +40,29 @@ module Lir = | Append of Expression * Expression | Let of SymbolId * string * bool * Expression * Expression | Fail of Expression + | DecisionTree of Expression * DecisionCase list + + and Pattern = + { Type: Type + Span: Span + Node: PatternNode } + + and PatternNode = + | Wildcard + | Binding of SymbolId * string + | LiteralPattern of Constant + | EmptyList + | ConsPattern of Pattern * Pattern + | TuplePattern of Pattern list + | RecordPattern of (string * Pattern) list + | SomePattern of Pattern + | NonePattern + + and DecisionCase = + { Pattern: Pattern + Guard: Expression option + Body: Expression + Span: Span } type Declaration = { Id: SymbolId @@ -74,6 +97,20 @@ module LirLowering = | Hir.Unit -> Lir.Unit | Hir.Int value -> Lir.Int value | Hir.Float value -> Lir.Float value | Hir.Bool value -> Lir.Bool value | Hir.String value -> Lir.String value + let rec private pattern (value: Hir.Pattern) : Lir.Pattern = + let node = + match value.Node with + | Hir.Wildcard -> Lir.Wildcard + | Hir.Binding(id, name) -> Lir.Binding(symbol id, name) + | Hir.LiteralPattern value -> Lir.LiteralPattern(constant value) + | Hir.EmptyList -> Lir.EmptyList + | Hir.ConsPattern(head, tail) -> Lir.ConsPattern(pattern head, pattern tail) + | Hir.TuplePattern items -> Lir.TuplePattern(List.map pattern items) + | Hir.RecordPattern fields -> Lir.RecordPattern(fields |> List.map (fun (name, item) -> name, pattern item)) + | Hir.SomePattern item -> Lir.SomePattern(pattern item) + | Hir.NonePattern -> Lir.NonePattern + { Type = value.Type; Span = value.Span; Node = node } + let rec private expression (value: Hir.Expression) : Lir.Expression = let node = match value.Node with @@ -93,6 +130,12 @@ module LirLowering = | Hir.Append(left, right) -> Lir.Append(expression left, expression right) | Hir.Let(id, name, recursive, binding, body) -> Lir.Let(symbol id, name, recursive, expression binding, expression body) | Hir.Fail value -> Lir.Fail(expression value) + | Hir.Match(target, cases) -> + Lir.DecisionTree(expression target, cases |> List.map (fun case -> + ({ Pattern = pattern case.Pattern + Guard = Option.map expression case.Guard + Body = expression case.Body + Span = case.Span }: Lir.DecisionCase))) { Type = value.Type; Span = value.Span; Node = node } let lower (program: Hir.Program) : Lir.Program = diff --git a/src/FScript.Compiler/LirInterpreter.fs b/src/FScript.Compiler/LirInterpreter.fs index 57673cb..04e5e2c 100644 --- a/src/FScript.Compiler/LirInterpreter.fs +++ b/src/FScript.Compiler/LirInterpreter.fs @@ -21,6 +21,10 @@ type LirValue = type ExecutionResult = { Value: LirValue } module LirInterpreter = + let private constantValue = function + | Lir.Unit -> VUnit | Lir.Int value -> VInt value | Lir.Float value -> VFloat value + | Lir.Bool value -> VBool value | Lir.String value -> VString value + let rec private equal left right = match left, right with | VUnit, VUnit -> true @@ -37,6 +41,27 @@ module LirInterpreter = let private fail code span message = Error [ { Code = code; Stage = Interpreter; Message = message; Span = Some span } ] + let rec private matchPattern (pattern: Lir.Pattern) value = + let combine left right = + match left, right with + | Some a, Some b -> Some(Map.fold (fun env key item -> Map.add key item env) a b) + | _ -> None + match pattern.Node, value with + | Lir.Wildcard, _ -> Some Map.empty + | Lir.Binding(id, _), item -> Some(Map.ofList [ id, item ]) + | Lir.LiteralPattern literal, item when equal (constantValue literal) item -> Some Map.empty + | Lir.EmptyList, VList [] -> Some Map.empty + | Lir.ConsPattern(head, tail), VList(headValue :: tailValues) -> + combine (matchPattern head headValue) (matchPattern tail (VList tailValues)) + | Lir.TuplePattern patterns, VTuple values when patterns.Length = values.Length -> + List.zip patterns values |> List.fold (fun state (itemPattern, itemValue) -> combine state (matchPattern itemPattern itemValue)) (Some Map.empty) + | Lir.RecordPattern patterns, VRecord values -> + patterns |> List.fold (fun state (name, itemPattern) -> + combine state (values |> Map.tryFind name |> Option.bind (matchPattern itemPattern))) (Some Map.empty) + | Lir.SomePattern itemPattern, VOption(Some item) -> matchPattern itemPattern item + | Lir.NonePattern, VOption None -> Some Map.empty + | _ -> None + let rec private evaluate environment (expression: Lir.Expression) = let binary work = match expression.Node with @@ -48,7 +73,7 @@ module LirInterpreter = | _ -> fail "FSI3002" expression.Span "Invalid primitive arity." match expression.Node with | Lir.Constant value -> - Ok(match value with Lir.Unit -> VUnit | Lir.Int v -> VInt v | Lir.Float v -> VFloat v | Lir.Bool v -> VBool v | Lir.String v -> VString v) + Ok(constantValue value) | Lir.Load id -> environment |> Map.tryFind id |> Option.map Ok |> Option.defaultWith (fun () -> fail "FSI3001" expression.Span (sprintf "Unknown LIR symbol %%%d." (Lir.symbolNumber id))) @@ -123,6 +148,25 @@ module LirInterpreter = | Ok(VString message) -> fail "FSI3011" expression.Span message | Ok _ -> fail "FSI3011" expression.Span "FScript program raised a non-string value." | Error errors -> Error errors + | Lir.DecisionTree(target, cases) -> + evaluate environment target |> Result.bind (fun targetValue -> + let rec run (remaining: Lir.DecisionCase list) = + match remaining with + | [] -> fail "FSI3012" expression.Span "Match failure." + | case :: rest -> + match matchPattern case.Pattern targetValue with + | None -> run rest + | Some bindings -> + let caseEnvironment = Map.fold (fun env id value -> Map.add id value env) environment bindings + match case.Guard with + | None -> evaluate caseEnvironment case.Body + | Some guard -> + match evaluate caseEnvironment guard with + | Ok(VBool true) -> evaluate caseEnvironment case.Body + | Ok(VBool false) -> run rest + | Ok _ -> fail "FSI3013" guard.Span "Match guard must evaluate to Boolean." + | Error errors -> Error errors + run cases) and private evaluateMany environment items = items |> List.map (evaluate environment) |> collect diff --git a/tests/FScript.Compiler.Tests/GoBackendTests.fs b/tests/FScript.Compiler.Tests/GoBackendTests.fs index fea128d..feb49bd 100644 --- a/tests/FScript.Compiler.Tests/GoBackendTests.fs +++ b/tests/FScript.Compiler.Tests/GoBackendTests.fs @@ -14,6 +14,8 @@ type GoBackendTests() = [] [] [] + [ x + 1\n | None -> 0", "4")>] + [ x\n | _ -> 42", "42")>] member _.``Generated projects run with Go`` (source: string, expected: string) = let input = { Program = Parser.parseProgram source; Externs = [] } let compiled = diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index 60d789e..4e4da7c 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -63,3 +63,15 @@ type HirTests() = | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSI3011")) Assert.That(diagnostics |> List.map _.Message, Does.Contain("boom")) + + [ x\n | [] -> 0", 1L)>] + [ x + 1\n | None -> 0", 4L)>] + [ x\n | _ -> 0", 1L)>] + [ x\n | _ -> 42", 42L)>] + member _.``Executes deterministic pattern decision trees`` (source: string, expected: int64) = + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt expected)) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index 2dec03e..e001df9 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -42,7 +42,7 @@ type IrArtifactTests() = [] member _.``IR artifact round trips lexical let and fail nodes`` () = - for source in [ "let f x = (let y = x + 1\n y\n)\nf 2"; "raise \"failure\"" ] do + for source in [ "let f x = (let y = x + 1\n y\n)\nf 2"; "raise \"failure\""; "match Some 2 with\n | Some x when x > 0 -> x\n | None -> 0" ] do let bytes = compile source |> _.Lir |> IrArtifact.serialize match IrArtifact.deserialize bytes with | Ok _ -> () From e5e1cba7c3000d19791634710e06988ba9a7e61b Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 16:57:43 +0200 Subject: [PATCH 011/133] feat(compiler): execute collection and data expressions --- CHANGELOG.md | 1 + docs/specs/compiler-semantics-baseline.md | 2 ++ src/FScript.Compiler/GoBackend.fs | 26 ++++++++++++-- src/FScript.Compiler/Hir.fs | 16 +++++++++ src/FScript.Compiler/HirLowering.fs | 23 ++++++++++++ src/FScript.Compiler/HirValidation.fs | 3 ++ src/FScript.Compiler/IrArtifact.fs | 32 +++++++++++++++++ src/FScript.Compiler/Lir.fs | 12 +++++++ src/FScript.Compiler/LirInterpreter.fs | 35 +++++++++++++++++++ .../FScript.Compiler.Tests/GoBackendTests.fs | 5 ++- tests/FScript.Compiler.Tests/HirTests.fs | 15 ++++++++ .../FScript.Compiler.Tests/IrArtifactTests.fs | 7 +++- 12 files changed, 173 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b86ff72..7f7e1ac 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added compiled ranges, safe list indexing, immutable record updates, interpolation, and FScript-style generated-program result formatting. - Added deterministic LIR pattern decision trees with single scrutinee evaluation, ordered guards, bindings, and match-failure diagnostics. - Added compiled local bindings, recursive lexical closures, and explicit runtime failures to HIR, LIR, and Go projects. - Added compiler, serialized LIR, interpreter, and Go support for tuples, records, options, lists, append, and recursive structural equality. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index ad8140d..40babc3 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -12,6 +12,8 @@ This document records the behavior that the typed IR, LIR interpreter, and compi - Top-level statements execute in source order. Every non-final expression must have type `unit`; the final expression is the program result. The first aggregate compiler slice preserves tuple field order, record field names, option presence, list element order, list cons/append behavior, and recursively structural equality through HIR, serialized LIR, interpretation, and Go generation. + +Ranges preserve inclusive ascending or descending order. List indexing returns an option and never throws for an out-of-range integer. Record updates allocate a new value, interpolation evaluates embedded expressions from left to right, and generated program output uses FScript value formatting rather than Go's default aggregate formatting. - Functions are curried. Multi-argument declarations are nested lambdas, application is one argument at a time, and closures capture their lexical environment. Single and mutual recursive function groups are supported. ## Types and values diff --git a/src/FScript.Compiler/GoBackend.fs b/src/FScript.Compiler/GoBackend.fs index 696e177..748147b 100644 --- a/src/FScript.Compiler/GoBackend.fs +++ b/src/FScript.Compiler/GoBackend.fs @@ -182,6 +182,28 @@ module GoBackend = match failure with | Some error -> Error error | None -> Ok $"func() {goType expression.Type} {{ matchValue := {targetCode}; {clauses}fsruntime.Fail(\"Match failure.\"); var zero {goType expression.Type}; return zero }}()") + | Lir.Range(startValue, endValue) -> + match emitExpression environment startValue, emitExpression environment endValue with + | Ok first, Ok last -> Ok $"fsruntime.Range({first}, {last})" + | Error error, _ | _, Error error -> Error error + | Lir.RecordUpdate(target, fields) -> + match emitExpression environment target, + (fields |> List.map (fun (name, item) -> emitExpression environment item |> Result.map (fun code -> System.Text.Json.JsonSerializer.Serialize(name) + ": " + code)) |> collect) with + | Ok targetCode, Ok values -> + let joined = String.concat ", " values + Ok $"fsruntime.UpdateRecord({targetCode}, map[string]any{{{joined}}})" + | Error error, _ | _, Error error -> Error error + | Lir.IndexGet(target, index) -> + match emitExpression environment target, emitExpression environment index with + | Ok targetCode, Ok indexCode -> Ok $"fsruntime.Index({targetCode}, {indexCode})" + | Error error, _ | _, Error error -> Error error + | Lir.Interpolate parts -> + parts |> List.map (function + | Lir.TextPart text -> Ok(System.Text.Json.JsonSerializer.Serialize(text)) + | Lir.ValuePart item -> emitExpression environment item) + |> collect |> Result.map (fun values -> + let joined = String.concat ", " values + $"fsruntime.Interpolate([]any{{{joined}}})") | Lir.Operation _ -> Error "Invalid operation arity" and private collect<'a> (results: Result<'a, string> list) : Result<'a list, string> = @@ -213,8 +235,8 @@ module GoBackend = let programSource = $"package fscriptprogram\n\nimport \"{moduleName}/internal/fsruntime\"\n\n{declarations}\nfunc Run() any {{ return {entryCode} }}\n" Ok [ { RelativePath = "go.mod"; Content = $"module {moduleName}\n\ngo 1.25\n" } - { RelativePath = "main.go"; Content = $"package main\n\nimport (\n\t\"fmt\"\n\t\"{moduleName}/internal/fscriptprogram\"\n)\n\nfunc main() {{ fmt.Println(fscriptprogram.Run()) }}\n" } + { RelativePath = "main.go"; Content = $"package main\n\nimport (\n\t\"fmt\"\n\t\"{moduleName}/internal/fscriptprogram\"\n\t\"{moduleName}/internal/fsruntime\"\n)\n\nfunc main() {{ fmt.Println(fsruntime.Format(fscriptprogram.Run())) }}\n" } { RelativePath = "internal/fscriptprogram/program.go"; Content = programSource } - { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n)\n\ntype Option struct { HasValue bool; Value any }\n\nfunc Int(value any) int64 { return value.(int64) }\nfunc Float(value any) float64 { return value.(float64) }\nfunc Bool(value any) bool { return value.(bool) }\nfunc String(value any) string { return value.(string) }\nfunc List(value any) []any { return value.([]any) }\nfunc Record(value any) map[string]any { return value.(map[string]any) }\nfunc AsOption(value any) Option { return value.(Option) }\nfunc Some(value any) Option { return Option{HasValue: true, Value: value} }\nfunc None() Option { return Option{} }\nfunc IsSome(value any) bool { return AsOption(value).HasValue }\nfunc OptionValue(value any) any { return AsOption(value).Value }\nfunc HasField(value any, name string) bool { _, ok := Record(value)[name]; return ok }\nfunc Equal(left any, right any) bool { return reflect.DeepEqual(left, right) }\nfunc Cons(head any, tail []any) []any { return append([]any{head}, tail...) }\nfunc Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) }\nfunc Tail(value any) []any { return append([]any{}, List(value)[1:]...) }\nfunc Fail(value any) { panic(fmt.Sprint(value)) }\n" } ] + { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"strings\"\n)\n\ntype Option struct { HasValue bool; Value any }\n\nfunc Int(value any) int64 { return value.(int64) }\nfunc Float(value any) float64 { return value.(float64) }\nfunc Bool(value any) bool { return value.(bool) }\nfunc String(value any) string { return value.(string) }\nfunc List(value any) []any { return value.([]any) }\nfunc Record(value any) map[string]any { return value.(map[string]any) }\nfunc AsOption(value any) Option { return value.(Option) }\nfunc Some(value any) Option { return Option{HasValue: true, Value: value} }\nfunc None() Option { return Option{} }\nfunc IsSome(value any) bool { return AsOption(value).HasValue }\nfunc OptionValue(value any) any { return AsOption(value).Value }\nfunc HasField(value any, name string) bool { _, ok := Record(value)[name]; return ok }\nfunc Equal(left any, right any) bool { return reflect.DeepEqual(left, right) }\nfunc Cons(head any, tail []any) []any { return append([]any{head}, tail...) }\nfunc Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) }\nfunc Tail(value any) []any { return append([]any{}, List(value)[1:]...) }\nfunc Range(first int64, last int64) []any { step := int64(1); if first > last { step = -1 }; values := []any{}; for current := first; ; current += step { values = append(values, current); if current == last { break } }; return values }\nfunc UpdateRecord(value any, updates map[string]any) map[string]any { result := map[string]any{}; for key, item := range Record(value) { result[key] = item }; for key, item := range updates { result[key] = item }; return result }\nfunc Index(value any, index any) Option { values := List(value); position := Int(index); if position < 0 || position >= int64(len(values)) { return None() }; return Some(values[position]) }\nfunc Format(value any) string { switch item := value.(type) { case bool: if item { return \"true\" }; return \"false\"; case Option: if !item.HasValue { return \"None\" }; return \"Some \" + Format(item.Value); case []any: parts := make([]string, len(item)); for index, value := range item { parts[index] = Format(value) }; return \"[\" + strings.Join(parts, \";\") + \"]\"; default: return fmt.Sprint(value) } }\nfunc Interpolate(values []any) string { var result strings.Builder; for _, value := range values { result.WriteString(Format(value)) }; return result.String() }\nfunc Fail(value any) { panic(fmt.Sprint(value)) }\n" } ] with ex -> Error [ { Code = "FSB4002"; Stage = GoGeneration; Message = ex.Message; Span = None } ] diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs index b079e43..9c92a4c 100644 --- a/src/FScript.Compiler/Hir.fs +++ b/src/FScript.Compiler/Hir.fs @@ -53,6 +53,14 @@ module Hir = | Let of SymbolId * string * bool * Expression * Expression | Fail of Expression | Match of Expression * MatchCase list + | Range of Expression * Expression + | RecordUpdate of Expression * (string * Expression) list + | IndexGet of Expression * Expression + | Interpolate of InterpolationPart list + + and InterpolationPart = + | TextPart of string + | ValuePart of Expression and Pattern = { Type: Type @@ -146,6 +154,14 @@ module Hir = lines.Add(typed $"match {cases.Length} cases") expression (indent + 2) target cases |> List.iter (fun case -> expression (indent + 2) case.Body) + | Range(startValue, endValue) -> lines.Add(typed "range"); expression (indent + 2) startValue; expression (indent + 2) endValue + | RecordUpdate(target, fields) -> + lines.Add(typed "record-update"); expression (indent + 2) target + fields |> List.iter (fun (_, item) -> expression (indent + 2) item) + | IndexGet(target, index) -> lines.Add(typed "index-get"); expression (indent + 2) target; expression (indent + 2) index + | Interpolate parts -> + lines.Add(typed "interpolate") + parts |> List.iter (function TextPart _ -> () | ValuePart item -> expression (indent + 2) item) for declaration in program.Declarations do lines.Add(sprintf "decl %%%d %s: %s" (symbolNumber declaration.Id) declaration.Name (formatType (let (Forall(_, value)) = declaration.Scheme in value))) diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index 0d219f7..fc30198 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -200,6 +200,29 @@ module HirLowering = | Ok loweredGuard, Ok loweredBody -> Ok({ Pattern = loweredPattern; Guard = loweredGuard; Body = loweredBody; Span = caseSpan }: Hir.MatchCase) | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e)) |> collectResults |> Result.map (fun loweredCases -> make (Hir.Match(loweredTarget, loweredCases))) + | ERange(startValue, endValue, _) -> + match lowerExpression state environment TInt startValue, lowerExpression state environment TInt endValue with + | Ok startItem, Ok endItem -> Ok(make (Hir.Range(startItem, endItem))) + | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e + | ERecordUpdate(target, fields, _) | EStructuralRecordUpdate(target, fields, _) -> + match lowerExpression state environment inferred target with + | Error errors -> Error errors + | Ok loweredTarget -> + let fieldTypes = match inferred with TRecord values -> values | _ -> Map.empty + fields |> List.map (fun (name, item) -> lowerExpression state environment (fieldTypes |> Map.tryFind name |> Option.defaultValue (Types.freshVar())) item |> Result.map (fun value -> name, value)) + |> collectResults |> Result.map (fun loweredFields -> make (Hir.RecordUpdate(loweredTarget, loweredFields))) + | EIndexGet(target, index, _) -> + let targetType = expressionType state target (Types.freshVar()) + let indexType = match targetType with TList _ -> TInt | TMap(key, _) -> key | _ -> Types.freshVar() + match lowerExpression state environment targetType target, lowerExpression state environment indexType index with + | Ok loweredTarget, Ok loweredIndex -> Ok(make (Hir.IndexGet(loweredTarget, loweredIndex))) + | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e + | ENameOf(name, _) -> Ok(make (Hir.Constant(Hir.String name))) + | EInterpolatedString(parts, _) -> + parts |> List.map (function + | IPText text -> Ok(Hir.TextPart text) + | IPExpr item -> lowerExpression state environment (expressionType state item (Types.freshVar())) item |> Result.map Hir.ValuePart) + |> collectResults |> Result.map (Hir.Interpolate >> make) | _ -> Error [ Diagnostic.hir "FSH1099" $"HIR lowering does not yet support {expression.GetType().Name}." (Some span) ] let lower (resolved: TypeInfer.ResolvedProgram) : Result = diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index 280e463..2dede89 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -41,6 +41,9 @@ module HirValidation = let values = Set.union (case.Guard |> Option.map references |> Option.defaultValue Set.empty) (references case.Body) Set.difference values bound Set.union (references target) (cases |> List.map caseReferences |> Set.unionMany) + | Hir.Range(startValue, endValue) | Hir.IndexGet(startValue, endValue) -> Set.union (references startValue) (references endValue) + | Hir.RecordUpdate(target, fields) -> Set.union (references target) (fields |> List.map (snd >> references) |> Set.unionMany) + | Hir.Interpolate parts -> parts |> List.choose (function Hir.TextPart _ -> None | Hir.ValuePart item -> Some(references item)) |> Set.unionMany let declared = declarationIds |> Set.ofList let unresolved = [ yield! program.Declarations |> List.collect (fun value -> references value.Value |> Set.toList) diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index f2306d2..ab657db 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -148,6 +148,27 @@ module IrArtifact = writer.WritePropertyName("body"); writeExpression writer case.Body; writer.WritePropertyName("span"); writeSpan writer case.Span writer.WriteEndObject() writer.WriteEndArray() + | Lir.Range(startValue, endValue) -> + writer.WriteString("kind", "range"); writer.WritePropertyName("start"); writeExpression writer startValue + writer.WritePropertyName("end"); writeExpression writer endValue + | Lir.RecordUpdate(target, fields) -> + writer.WriteString("kind", "record-update"); writer.WritePropertyName("target"); writeExpression writer target + writer.WriteStartArray("fields") + for name, field in fields do + writer.WriteStartObject(); writer.WriteString("name", name); writer.WritePropertyName("value"); writeExpression writer field; writer.WriteEndObject() + writer.WriteEndArray() + | Lir.IndexGet(target, index) -> + writer.WriteString("kind", "index-get"); writer.WritePropertyName("target"); writeExpression writer target + writer.WritePropertyName("index"); writeExpression writer index + | Lir.Interpolate parts -> + writer.WriteString("kind", "interpolate"); writer.WriteStartArray("parts") + for part in parts do + writer.WriteStartObject() + match part with + | Lir.TextPart text -> writer.WriteString("kind", "text"); writer.WriteString("text", text) + | Lir.ValuePart item -> writer.WriteString("kind", "value"); writer.WritePropertyName("value"); writeExpression writer item + writer.WriteEndObject() + writer.WriteEndArray() writer.WriteEndObject() let serialize (program: Lir.Program) = @@ -256,6 +277,17 @@ module IrArtifact = Body = readExpression (property "body" item) Span = readSpan (property "span" item) }: Lir.DecisionCase)) |> Seq.toList Lir.DecisionTree(readExpression (property "target" element), cases) + | "range" -> Lir.Range(readExpression (property "start" element), readExpression (property "end" element)) + | "record-update" -> + let fields = (property "fields" element).EnumerateArray() |> Seq.map (fun item -> stringProperty "name" item, readExpression (property "value" item)) |> Seq.toList + Lir.RecordUpdate(readExpression (property "target" element), fields) + | "index-get" -> Lir.IndexGet(readExpression (property "target" element), readExpression (property "index" element)) + | "interpolate" -> + Lir.Interpolate((property "parts" element).EnumerateArray() |> Seq.map (fun item -> + match stringProperty "kind" item with + | "text" -> Lir.TextPart(stringProperty "text" item) + | "value" -> Lir.ValuePart(readExpression (property "value" item)) + | kind -> invalidOp $"Unknown interpolation part '{kind}'.") |> Seq.toList) | kind -> invalidOp $"Unknown LIR expression kind '{kind}'." { Type = expressionType; Span = span; Node = node } diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index 9a9ec85..b5e5cac 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -41,6 +41,14 @@ module Lir = | Let of SymbolId * string * bool * Expression * Expression | Fail of Expression | DecisionTree of Expression * DecisionCase list + | Range of Expression * Expression + | RecordUpdate of Expression * (string * Expression) list + | IndexGet of Expression * Expression + | Interpolate of InterpolationPart list + + and InterpolationPart = + | TextPart of string + | ValuePart of Expression and Pattern = { Type: Type @@ -136,6 +144,10 @@ module LirLowering = Guard = Option.map expression case.Guard Body = expression case.Body Span = case.Span }: Lir.DecisionCase))) + | Hir.Range(startValue, endValue) -> Lir.Range(expression startValue, expression endValue) + | Hir.RecordUpdate(target, fields) -> Lir.RecordUpdate(expression target, fields |> List.map (fun (name, item) -> name, expression item)) + | Hir.IndexGet(target, index) -> Lir.IndexGet(expression target, expression index) + | Hir.Interpolate parts -> Lir.Interpolate(parts |> List.map (function Hir.TextPart text -> Lir.TextPart text | Hir.ValuePart item -> Lir.ValuePart(expression item))) { Type = value.Type; Span = value.Span; Node = node } let lower (program: Hir.Program) : Lir.Program = diff --git a/src/FScript.Compiler/LirInterpreter.fs b/src/FScript.Compiler/LirInterpreter.fs index 04e5e2c..b09ed8c 100644 --- a/src/FScript.Compiler/LirInterpreter.fs +++ b/src/FScript.Compiler/LirInterpreter.fs @@ -38,6 +38,17 @@ module LirInterpreter = | VOption(Some a), VOption(Some b) -> equal a b | _ -> false + let rec private formatValue = function + | VUnit -> "()" | VInt value -> string value | VFloat value -> value.ToString("R", System.Globalization.CultureInfo.InvariantCulture) + | VBool value -> value.ToString().ToLowerInvariant() | VString value -> value + | VTuple values -> "(" + (values |> List.map formatValue |> String.concat ", ") + ")" + | VList values -> "[" + (values |> List.map formatValue |> String.concat ";") + "]" + | VRecord fields -> + let rendered = fields |> Map.toList |> List.map (fun (name, value) -> name + " = " + formatValue value) |> String.concat "; " + "{ " + rendered + " }" + | VOption None -> "None" | VOption(Some value) -> $"Some {formatValue value}" + | VClosure _ -> "" + let private fail code span message = Error [ { Code = code; Stage = Interpreter; Message = message; Span = Some span } ] @@ -167,6 +178,30 @@ module LirInterpreter = | Ok _ -> fail "FSI3013" guard.Span "Match guard must evaluate to Boolean." | Error errors -> Error errors run cases) + | Lir.Range(startValue, endValue) -> + match evaluate environment startValue, evaluate environment endValue with + | Ok(VInt first), Ok(VInt last) -> + let step = if first <= last then 1L else -1L + Ok(VList([ first .. step .. last ] |> List.map VInt)) + | Ok _, Ok _ -> fail "FSI3014" expression.Span "Range endpoints must be integers." + | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e + | Lir.RecordUpdate(target, fields) -> + match evaluate environment target with + | Ok(VRecord existing) -> + fields |> List.map (fun (name, item) -> evaluate environment item |> Result.map (fun value -> name, value)) + |> collect |> Result.map (fun updates -> VRecord(List.fold (fun state (name, value) -> Map.add name value state) existing updates)) + | Ok _ -> fail "FSI3015" expression.Span "Record update requires a record." + | Error errors -> Error errors + | Lir.IndexGet(target, index) -> + match evaluate environment target, evaluate environment index with + | Ok(VList values), Ok(VInt position) -> + if position < 0L || position >= int64 values.Length then Ok(VOption None) + else Ok(VOption(Some values.[int position])) + | Ok _, Ok _ -> fail "FSI3016" expression.Span "Index access requires a list and integer index." + | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e + | Lir.Interpolate parts -> + parts |> List.map (function Lir.TextPart text -> Ok text | Lir.ValuePart item -> evaluate environment item |> Result.map formatValue) + |> collect |> Result.map (String.concat "" >> VString) and private evaluateMany environment items = items |> List.map (evaluate environment) |> collect diff --git a/tests/FScript.Compiler.Tests/GoBackendTests.fs b/tests/FScript.Compiler.Tests/GoBackendTests.fs index feb49bd..9cad974 100644 --- a/tests/FScript.Compiler.Tests/GoBackendTests.fs +++ b/tests/FScript.Compiler.Tests/GoBackendTests.fs @@ -10,12 +10,15 @@ open NUnit.Framework [] type GoBackendTests() = [] - [] + [] [] [] [] [ x + 1\n | None -> 0", "4")>] [ x\n | _ -> 42", "42")>] + [] + [] + [] member _.``Generated projects run with Go`` (source: string, expected: string) = let input = { Program = Parser.parseProgram source; Externs = [] } let compiled = diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index 4e4da7c..7e17760 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -75,3 +75,18 @@ type HirTests() = match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt expected)) | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``Executes ranges indexing updates and interpolation`` () = + let cases = + [ "[1..3]", LirValue.VList [ LirValue.VInt 1L; LirValue.VInt 2L; LirValue.VInt 3L ] + "[1; 2][1]", LirValue.VOption(Some(LirValue.VInt 2L)) + "let value = {| Count = 1 |}\n{| value with Count = 2 |}.Count", LirValue.VInt 2L + "$\"value {1 + 2}\"", LirValue.VString "value 3" ] + for source, expected in cases do + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(expected), source) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index e001df9..e724c79 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -42,7 +42,12 @@ type IrArtifactTests() = [] member _.``IR artifact round trips lexical let and fail nodes`` () = - for source in [ "let f x = (let y = x + 1\n y\n)\nf 2"; "raise \"failure\""; "match Some 2 with\n | Some x when x > 0 -> x\n | None -> 0" ] do + for source in + [ "let f x = (let y = x + 1\n y\n)\nf 2" + "raise \"failure\"" + "match Some 2 with\n | Some x when x > 0 -> x\n | None -> 0" + "([1..3][0], $\"answer {42}\")" + "let value = {| Count = 1 |}\n{| value with Count = 2 |}" ] do let bytes = compile source |> _.Lir |> IrArtifact.serialize match IrArtifact.deserialize bytes with | Ok _ -> () From 3eb070c71b1ed70560912c5fcbcaa38cce66d6b9 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 17:01:31 +0200 Subject: [PATCH 012/133] feat(compiler): preserve declared unions --- CHANGELOG.md | 1 + docs/specs/compiler-semantics-baseline.md | 2 + src/FScript.Compiler/GoBackend.fs | 19 ++++++- src/FScript.Compiler/Hir.fs | 5 ++ src/FScript.Compiler/HirLowering.fs | 51 ++++++++++++++++++- src/FScript.Compiler/HirValidation.fs | 2 + src/FScript.Compiler/IrArtifact.fs | 12 +++++ src/FScript.Compiler/Lir.fs | 4 ++ src/FScript.Compiler/LirInterpreter.fs | 8 +++ .../FScript.Compiler.Tests/GoBackendTests.fs | 1 + tests/FScript.Compiler.Tests/HirTests.fs | 10 ++++ .../FScript.Compiler.Tests/IrArtifactTests.fs | 3 +- 12 files changed, 114 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7f7e1ac..20d9d65 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added declared-union constructors, payloads, structural equality, pattern matching, artifact serialization, and Go runtime representation. - Added compiled ranges, safe list indexing, immutable record updates, interpolation, and FScript-style generated-program result formatting. - Added deterministic LIR pattern decision trees with single scrutinee evaluation, ordered guards, bindings, and match-failure diagnostics. - Added compiled local bindings, recursive lexical closures, and explicit runtime failures to HIR, LIR, and Go projects. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 40babc3..78ff69e 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -14,6 +14,8 @@ This document records the behavior that the typed IR, LIR interpreter, and compi The first aggregate compiler slice preserves tuple field order, record field names, option presence, list element order, list cons/append behavior, and recursively structural equality through HIR, serialized LIR, interpretation, and Go generation. Ranges preserve inclusive ascending or descending order. List indexing returns an option and never throws for an out-of-range integer. Record updates allocate a new value, interpolation evaluates embedded expressions from left to right, and generated program output uses FScript value formatting rather than Go's default aggregate formatting. + +Declared unions retain their type and case identity plus an optional erased payload. Constructors are deterministic compiler declarations, qualified and unqualified case references share one identity, pattern tests check both type and case tags, and equality recursively compares payloads. - Functions are curried. Multi-argument declarations are nested lambdas, application is one argument at a time, and closures capture their lexical environment. Single and mutual recursive function groups are supported. ## Types and values diff --git a/src/FScript.Compiler/GoBackend.fs b/src/FScript.Compiler/GoBackend.fs index 748147b..ad4993e 100644 --- a/src/FScript.Compiler/GoBackend.fs +++ b/src/FScript.Compiler/GoBackend.fs @@ -20,6 +20,7 @@ module GoBackend = | TList _ -> "[]any" | TOption _ -> "fsruntime.Option" | TRecord _ -> "map[string]any" + | TUnion _ | TNamed _ -> "fsruntime.Union" | value -> invalidArg "value" $"Unsupported Go representation for {Types.typeToString value}." let private identifier prefix id = $"{prefix}{Lir.symbolNumber id}" @@ -55,6 +56,7 @@ module GoBackend = | TList _ | TTuple _ -> $"fsruntime.List({code})" | TRecord _ -> $"fsruntime.Record({code})" | TOption _ -> $"fsruntime.AsOption({code})" + | TUnion _ | TNamed _ -> $"fsruntime.AsUnion({code})" | _ -> code let rec private patternBindings (pattern: Lir.Pattern) = @@ -64,6 +66,7 @@ module GoBackend = | Lir.TuplePattern items -> List.collect patternBindings items | Lir.RecordPattern fields -> fields |> List.collect (snd >> patternBindings) | Lir.SomePattern item -> patternBindings item + | Lir.UnionPattern(_, _, payload) -> payload |> Option.map patternBindings |> Option.defaultValue [] | _ -> [] let rec private emitPattern valueCode (pattern: Lir.Pattern) = @@ -96,6 +99,14 @@ module GoBackend = let test, assignments = emitPattern $"fsruntime.Record({valueCode})[{quoted}]" item $"fsruntime.HasField({valueCode}, {quoted}) && ({test})", assignments) |> combine + | Lir.UnionPattern(typeName, caseName, payload) -> + let quotedType = System.Text.Json.JsonSerializer.Serialize(typeName) + let quotedCase = System.Text.Json.JsonSerializer.Serialize(caseName) + match payload with + | None -> $"fsruntime.IsUnion({valueCode}, {quotedType}, {quotedCase}) && !fsruntime.UnionHasValue({valueCode})", [] + | Some item -> + let test, assignments = emitPattern $"fsruntime.UnionValue({valueCode})" item + $"fsruntime.IsUnion({valueCode}, {quotedType}, {quotedCase}) && fsruntime.UnionHasValue({valueCode}) && ({test})", assignments let rec private emitExpression environment (expression: Lir.Expression) = match expression.Node with @@ -204,6 +215,12 @@ module GoBackend = |> collect |> Result.map (fun values -> let joined = String.concat ", " values $"fsruntime.Interpolate([]any{{{joined}}})") + | Lir.UnionCase(typeName, caseName, payload) -> + let quotedType = System.Text.Json.JsonSerializer.Serialize(typeName) + let quotedCase = System.Text.Json.JsonSerializer.Serialize(caseName) + match payload with + | None -> Ok $"fsruntime.MakeUnion({quotedType}, {quotedCase})" + | Some item -> emitExpression environment item |> Result.map (fun code -> $"fsruntime.MakeUnionValue({quotedType}, {quotedCase}, {code})") | Lir.Operation _ -> Error "Invalid operation arity" and private collect<'a> (results: Result<'a, string> list) : Result<'a list, string> = @@ -237,6 +254,6 @@ module GoBackend = [ { RelativePath = "go.mod"; Content = $"module {moduleName}\n\ngo 1.25\n" } { RelativePath = "main.go"; Content = $"package main\n\nimport (\n\t\"fmt\"\n\t\"{moduleName}/internal/fscriptprogram\"\n\t\"{moduleName}/internal/fsruntime\"\n)\n\nfunc main() {{ fmt.Println(fsruntime.Format(fscriptprogram.Run())) }}\n" } { RelativePath = "internal/fscriptprogram/program.go"; Content = programSource } - { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"strings\"\n)\n\ntype Option struct { HasValue bool; Value any }\n\nfunc Int(value any) int64 { return value.(int64) }\nfunc Float(value any) float64 { return value.(float64) }\nfunc Bool(value any) bool { return value.(bool) }\nfunc String(value any) string { return value.(string) }\nfunc List(value any) []any { return value.([]any) }\nfunc Record(value any) map[string]any { return value.(map[string]any) }\nfunc AsOption(value any) Option { return value.(Option) }\nfunc Some(value any) Option { return Option{HasValue: true, Value: value} }\nfunc None() Option { return Option{} }\nfunc IsSome(value any) bool { return AsOption(value).HasValue }\nfunc OptionValue(value any) any { return AsOption(value).Value }\nfunc HasField(value any, name string) bool { _, ok := Record(value)[name]; return ok }\nfunc Equal(left any, right any) bool { return reflect.DeepEqual(left, right) }\nfunc Cons(head any, tail []any) []any { return append([]any{head}, tail...) }\nfunc Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) }\nfunc Tail(value any) []any { return append([]any{}, List(value)[1:]...) }\nfunc Range(first int64, last int64) []any { step := int64(1); if first > last { step = -1 }; values := []any{}; for current := first; ; current += step { values = append(values, current); if current == last { break } }; return values }\nfunc UpdateRecord(value any, updates map[string]any) map[string]any { result := map[string]any{}; for key, item := range Record(value) { result[key] = item }; for key, item := range updates { result[key] = item }; return result }\nfunc Index(value any, index any) Option { values := List(value); position := Int(index); if position < 0 || position >= int64(len(values)) { return None() }; return Some(values[position]) }\nfunc Format(value any) string { switch item := value.(type) { case bool: if item { return \"true\" }; return \"false\"; case Option: if !item.HasValue { return \"None\" }; return \"Some \" + Format(item.Value); case []any: parts := make([]string, len(item)); for index, value := range item { parts[index] = Format(value) }; return \"[\" + strings.Join(parts, \";\") + \"]\"; default: return fmt.Sprint(value) } }\nfunc Interpolate(values []any) string { var result strings.Builder; for _, value := range values { result.WriteString(Format(value)) }; return result.String() }\nfunc Fail(value any) { panic(fmt.Sprint(value)) }\n" } ] + { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"strings\"\n)\n\ntype Option struct { HasValue bool; Value any }\ntype Union struct { TypeName string; CaseName string; HasValue bool; Value any }\n\nfunc Int(value any) int64 { return value.(int64) }\nfunc Float(value any) float64 { return value.(float64) }\nfunc Bool(value any) bool { return value.(bool) }\nfunc String(value any) string { return value.(string) }\nfunc List(value any) []any { return value.([]any) }\nfunc Record(value any) map[string]any { return value.(map[string]any) }\nfunc AsOption(value any) Option { return value.(Option) }\nfunc AsUnion(value any) Union { return value.(Union) }\nfunc Some(value any) Option { return Option{HasValue: true, Value: value} }\nfunc None() Option { return Option{} }\nfunc IsSome(value any) bool { return AsOption(value).HasValue }\nfunc OptionValue(value any) any { return AsOption(value).Value }\nfunc MakeUnion(typeName string, caseName string) Union { return Union{TypeName: typeName, CaseName: caseName} }\nfunc MakeUnionValue(typeName string, caseName string, value any) Union { return Union{TypeName: typeName, CaseName: caseName, HasValue: true, Value: value} }\nfunc IsUnion(value any, typeName string, caseName string) bool { item := AsUnion(value); return item.TypeName == typeName && item.CaseName == caseName }\nfunc UnionHasValue(value any) bool { return AsUnion(value).HasValue }\nfunc UnionValue(value any) any { return AsUnion(value).Value }\nfunc HasField(value any, name string) bool { _, ok := Record(value)[name]; return ok }\nfunc Equal(left any, right any) bool { return reflect.DeepEqual(left, right) }\nfunc Cons(head any, tail []any) []any { return append([]any{head}, tail...) }\nfunc Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) }\nfunc Tail(value any) []any { return append([]any{}, List(value)[1:]...) }\nfunc Range(first int64, last int64) []any { step := int64(1); if first > last { step = -1 }; values := []any{}; for current := first; ; current += step { values = append(values, current); if current == last { break } }; return values }\nfunc UpdateRecord(value any, updates map[string]any) map[string]any { result := map[string]any{}; for key, item := range Record(value) { result[key] = item }; for key, item := range updates { result[key] = item }; return result }\nfunc Index(value any, index any) Option { values := List(value); position := Int(index); if position < 0 || position >= int64(len(values)) { return None() }; return Some(values[position]) }\nfunc Format(value any) string { switch item := value.(type) { case bool: if item { return \"true\" }; return \"false\"; case Option: if !item.HasValue { return \"None\" }; return \"Some \" + Format(item.Value); case Union: if !item.HasValue { return item.CaseName }; return item.CaseName + \" \" + Format(item.Value); case []any: parts := make([]string, len(item)); for index, value := range item { parts[index] = Format(value) }; return \"[\" + strings.Join(parts, \";\") + \"]\"; default: return fmt.Sprint(value) } }\nfunc Interpolate(values []any) string { var result strings.Builder; for _, value := range values { result.WriteString(Format(value)) }; return result.String() }\nfunc Fail(value any) { panic(fmt.Sprint(value)) }\n" } ] with ex -> Error [ { Code = "FSB4002"; Stage = GoGeneration; Message = ex.Message; Span = None } ] diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs index 9c92a4c..0db4c34 100644 --- a/src/FScript.Compiler/Hir.fs +++ b/src/FScript.Compiler/Hir.fs @@ -57,6 +57,7 @@ module Hir = | RecordUpdate of Expression * (string * Expression) list | IndexGet of Expression * Expression | Interpolate of InterpolationPart list + | UnionCase of string * string * Expression option and InterpolationPart = | TextPart of string @@ -77,6 +78,7 @@ module Hir = | RecordPattern of (string * Pattern) list | SomePattern of Pattern | NonePattern + | UnionPattern of string * string * Pattern option and MatchCase = { Pattern: Pattern @@ -162,6 +164,9 @@ module Hir = | Interpolate parts -> lines.Add(typed "interpolate") parts |> List.iter (function TextPart _ -> () | ValuePart item -> expression (indent + 2) item) + | UnionCase(typeName, caseName, payload) -> + lines.Add(typed $"union {typeName}.{caseName}") + payload |> Option.iter (expression (indent + 2)) for declaration in program.Declarations do lines.Add(sprintf "decl %%%d %s: %s" (symbolNumber declaration.Id) declaration.Name (formatType (let (Forall(_, value)) = declaration.Scheme in value))) diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index fc30198..9cd9d5c 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -6,6 +6,7 @@ module HirLowering = type private State = { mutable NextSymbol: int ExpressionTypes: Map + mutable UnionCases: Map mutable Diagnostics: CompilerDiagnostic list } let private newSymbol state = @@ -83,6 +84,17 @@ module HirLowering = let fields, bindingMaps = List.unzip values let bindings = bindingMaps |> List.fold (Map.fold (fun env key value -> Map.add key value env)) Map.empty make (Hir.RecordPattern fields), bindings) + | PUnionCase(typeName, caseName, payload, _) -> + let resolvedTypeName, payloadType = + match expected with + | TUnion(name, cases) -> name, cases |> Map.tryFind caseName |> Option.flatten + | _ -> + state.UnionCases |> Map.tryFind caseName + |> Option.defaultValue (typeName |> Option.defaultValue "", None) + match payload, payloadType with + | None, _ -> Ok(make (Hir.UnionPattern(resolvedTypeName, caseName, None)), Map.empty) + | Some item, Some itemType -> lowerPattern state itemType item |> Result.map (fun (value, bindings) -> make (Hir.UnionPattern(resolvedTypeName, caseName, Some value)), bindings) + | Some item, None -> lowerPattern state (Types.freshVar()) item |> Result.map (fun (value, bindings) -> make (Hir.UnionPattern(resolvedTypeName, caseName, Some value)), bindings) | _ -> Error [ Diagnostic.hir "FSH1004" "Pattern form is not yet supported by HIR lowering." (Some span) ] let rec private lowerExpression state environment fallback expression = @@ -229,13 +241,35 @@ module HirLowering = let expressionTypes = resolved.ExpressionTypes |> List.fold (fun state item -> state |> Map.add item.Expr item.Type) Map.empty - let state = { NextSymbol = 0; ExpressionTypes = expressionTypes; Diagnostics = [] } + let state = { NextSymbol = 0; ExpressionTypes = expressionTypes; UnionCases = Map.empty; Diagnostics = [] } let builtins = BuiltinSignatures.builtinSchemes |> Map.toList |> List.sortBy fst |> List.map (fun (name, _) -> name, newSymbol state) |> Map.ofList + let typeDefinitions = + resolved.TypedProgram |> List.choose (function TypeInfer.TSType definition -> Some definition | _ -> None) + let rec typeFromRef = function + | TRName "unit" -> TUnit | TRName "int" -> TInt | TRName "float" -> TFloat | TRName "bool" -> TBool | TRName "string" -> TString + | TRName name -> TNamed name + | TRTuple items -> TTuple(List.map typeFromRef items) + | TRFun(argument, result) -> TFun(typeFromRef argument, typeFromRef result) + | TRPostfix(item, "list") -> TList(typeFromRef item) + | TRPostfix(item, "option") -> TOption(typeFromRef item) + | TRPostfix(item, "task") -> TTask(typeFromRef item) + | TRRecord fields | TRStructuralRecord fields -> TRecord(fields |> List.map (fun (name, item) -> name, typeFromRef item) |> Map.ofList) + | value -> TNamed(sprintf "%A" value) + let unionDefinitions = + typeDefinitions |> List.filter (fun definition -> not definition.Cases.IsEmpty) |> List.map (fun definition -> + let cases = definition.Cases |> List.map (fun (name, payload) -> name, Option.map typeFromRef payload) |> Map.ofList + definition, TUnion(definition.Name, cases)) + state.UnionCases <- + unionDefinitions |> List.collect (fun (definition, _) -> + let cases = definition.Cases |> List.map (fun (name, payload) -> name, Option.map typeFromRef payload) |> Map.ofList + definition.Cases |> List.map (fun (caseName, _) -> caseName, (definition.Name, cases.[caseName]))) |> Map.ofList + let constructorSymbols = + unionDefinitions |> List.collect (fun (definition, _) -> definition.Cases |> List.map (fun (caseName, _) -> definition.Name, caseName, newSymbol state)) let topLevel = resolved.TypedProgram |> List.collect (function @@ -244,8 +278,21 @@ module HirLowering = | _ -> []) |> List.distinct |> List.fold (fun env name -> env |> Map.add name (newSymbol state)) builtins + let topLevel = + constructorSymbols |> List.fold (fun env (typeName, caseName, id) -> env |> Map.add caseName id |> Map.add $"{typeName}.{caseName}" id) topLevel - let mutable declarations = [] + let mutable declarations = + constructorSymbols |> List.map (fun (typeName, caseName, id) -> + let definition, unionType = unionDefinitions |> List.find (fun (definition, _) -> definition.Name = typeName) + let payloadType = definition.Cases |> List.find (fst >> (=) caseName) |> snd |> Option.map typeFromRef + let value = + match payloadType with + | None -> ({ Type = unionType; Span = definition.Span; Node = Hir.UnionCase(typeName, caseName, None) }: Hir.Expression) + | Some itemType -> + let parameter = newSymbol state + let body = ({ Type = unionType; Span = definition.Span; Node = Hir.UnionCase(typeName, caseName, Some({ Type = itemType; Span = definition.Span; Node = Hir.Reference parameter })) }: Hir.Expression) + ({ Type = TFun(itemType, unionType); Span = definition.Span; Node = Hir.Lambda(parameter, "value", itemType, body) }: Hir.Expression) + ({ Id = id; Name = $"{typeName}.{caseName}"; Scheme = Types.generalize Map.empty value.Type; IsRecursive = false; IsExported = false; Span = definition.Span; Value = value }: Hir.Declaration)) let mutable entry = None let mutable errors = [] for statement in resolved.TypedProgram do diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index 2dede89..a14fb26 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -35,6 +35,7 @@ module HirValidation = | Hir.TuplePattern items -> items |> List.map bindings |> Set.unionMany | Hir.RecordPattern fields -> fields |> List.map (snd >> bindings) |> Set.unionMany | Hir.SomePattern item -> bindings item + | Hir.UnionPattern(_, _, payload) -> payload |> Option.map bindings |> Option.defaultValue Set.empty | _ -> Set.empty let caseReferences (case: Hir.MatchCase) = let bound = bindings case.Pattern @@ -44,6 +45,7 @@ module HirValidation = | Hir.Range(startValue, endValue) | Hir.IndexGet(startValue, endValue) -> Set.union (references startValue) (references endValue) | Hir.RecordUpdate(target, fields) -> Set.union (references target) (fields |> List.map (snd >> references) |> Set.unionMany) | Hir.Interpolate parts -> parts |> List.choose (function Hir.TextPart _ -> None | Hir.ValuePart item -> Some(references item)) |> Set.unionMany + | Hir.UnionCase(_, _, payload) -> payload |> Option.map references |> Option.defaultValue Set.empty let declared = declarationIds |> Set.ofList let unresolved = [ yield! program.Declarations |> List.collect (fun value -> references value.Value |> Set.toList) diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index ab657db..821a4e8 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -91,6 +91,9 @@ module IrArtifact = for name, item in fields do writer.WriteStartObject(); writer.WriteString("name", name); writer.WritePropertyName("pattern"); writePattern writer item; writer.WriteEndObject() writer.WriteEndArray() + | Lir.UnionPattern(typeName, caseName, payload) -> + writer.WriteString("kind", "union"); writer.WriteString("typeName", typeName); writer.WriteString("caseName", caseName) + match payload with Some item -> writer.WritePropertyName("payload"); writePattern writer item | None -> writer.WriteNull("payload") writer.WriteEndObject() let rec private writeExpression (writer: Utf8JsonWriter) (value: Lir.Expression) = @@ -169,6 +172,9 @@ module IrArtifact = | Lir.ValuePart item -> writer.WriteString("kind", "value"); writer.WritePropertyName("value"); writeExpression writer item writer.WriteEndObject() writer.WriteEndArray() + | Lir.UnionCase(typeName, caseName, payload) -> + writer.WriteString("kind", "union-case"); writer.WriteString("typeName", typeName); writer.WriteString("caseName", caseName) + match payload with Some item -> writer.WritePropertyName("payload"); writeExpression writer item | None -> writer.WriteNull("payload") writer.WriteEndObject() let serialize (program: Lir.Program) = @@ -242,6 +248,9 @@ module IrArtifact = | "cons" -> Lir.ConsPattern(readPattern (property "head" element), readPattern (property "tail" element)) | "tuple" -> Lir.TuplePattern((property "items" element).EnumerateArray() |> Seq.map readPattern |> Seq.toList) | "record" -> Lir.RecordPattern((property "fields" element).EnumerateArray() |> Seq.map (fun field -> stringProperty "name" field, readPattern (property "pattern" field)) |> Seq.toList) + | "union" -> + let payload = property "payload" element + Lir.UnionPattern(stringProperty "typeName" element, stringProperty "caseName" element, if payload.ValueKind = JsonValueKind.Null then None else Some(readPattern payload)) | kind -> invalidOp $"Unknown LIR pattern kind '{kind}'." { Type = readType (property "type" element); Span = readSpan (property "span" element); Node = node } @@ -288,6 +297,9 @@ module IrArtifact = | "text" -> Lir.TextPart(stringProperty "text" item) | "value" -> Lir.ValuePart(readExpression (property "value" item)) | kind -> invalidOp $"Unknown interpolation part '{kind}'.") |> Seq.toList) + | "union-case" -> + let payload = property "payload" element + Lir.UnionCase(stringProperty "typeName" element, stringProperty "caseName" element, if payload.ValueKind = JsonValueKind.Null then None else Some(readExpression payload)) | kind -> invalidOp $"Unknown LIR expression kind '{kind}'." { Type = expressionType; Span = span; Node = node } diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index b5e5cac..0b77d5e 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -45,6 +45,7 @@ module Lir = | RecordUpdate of Expression * (string * Expression) list | IndexGet of Expression * Expression | Interpolate of InterpolationPart list + | UnionCase of string * string * Expression option and InterpolationPart = | TextPart of string @@ -65,6 +66,7 @@ module Lir = | RecordPattern of (string * Pattern) list | SomePattern of Pattern | NonePattern + | UnionPattern of string * string * Pattern option and DecisionCase = { Pattern: Pattern @@ -117,6 +119,7 @@ module LirLowering = | Hir.RecordPattern fields -> Lir.RecordPattern(fields |> List.map (fun (name, item) -> name, pattern item)) | Hir.SomePattern item -> Lir.SomePattern(pattern item) | Hir.NonePattern -> Lir.NonePattern + | Hir.UnionPattern(typeName, caseName, payload) -> Lir.UnionPattern(typeName, caseName, Option.map pattern payload) { Type = value.Type; Span = value.Span; Node = node } let rec private expression (value: Hir.Expression) : Lir.Expression = @@ -148,6 +151,7 @@ module LirLowering = | Hir.RecordUpdate(target, fields) -> Lir.RecordUpdate(expression target, fields |> List.map (fun (name, item) -> name, expression item)) | Hir.IndexGet(target, index) -> Lir.IndexGet(expression target, expression index) | Hir.Interpolate parts -> Lir.Interpolate(parts |> List.map (function Hir.TextPart text -> Lir.TextPart text | Hir.ValuePart item -> Lir.ValuePart(expression item))) + | Hir.UnionCase(typeName, caseName, payload) -> Lir.UnionCase(typeName, caseName, Option.map expression payload) { Type = value.Type; Span = value.Span; Node = node } let lower (program: Hir.Program) : Lir.Program = diff --git a/src/FScript.Compiler/LirInterpreter.fs b/src/FScript.Compiler/LirInterpreter.fs index b09ed8c..c3457d6 100644 --- a/src/FScript.Compiler/LirInterpreter.fs +++ b/src/FScript.Compiler/LirInterpreter.fs @@ -16,6 +16,7 @@ type LirValue = | VList of LirValue list | VRecord of Map | VOption of LirValue option + | VUnion of string * string * LirValue option | VClosure of Lir.SymbolId * string * Lir.Expression * Map ref type ExecutionResult = { Value: LirValue } @@ -36,6 +37,7 @@ module LirInterpreter = | VRecord a, VRecord b -> a.Count = b.Count && (a |> Map.forall (fun key value -> b |> Map.tryFind key |> Option.exists (equal value))) | VOption None, VOption None -> true | VOption(Some a), VOption(Some b) -> equal a b + | VUnion(aType, aCase, aValue), VUnion(bType, bCase, bValue) -> aType = bType && aCase = bCase && equal (VOption aValue) (VOption bValue) | _ -> false let rec private formatValue = function @@ -47,6 +49,7 @@ module LirInterpreter = let rendered = fields |> Map.toList |> List.map (fun (name, value) -> name + " = " + formatValue value) |> String.concat "; " "{ " + rendered + " }" | VOption None -> "None" | VOption(Some value) -> $"Some {formatValue value}" + | VUnion(_, caseName, None) -> caseName | VUnion(_, caseName, Some value) -> $"{caseName} {formatValue value}" | VClosure _ -> "" let private fail code span message = @@ -71,6 +74,8 @@ module LirInterpreter = combine state (values |> Map.tryFind name |> Option.bind (matchPattern itemPattern))) (Some Map.empty) | Lir.SomePattern itemPattern, VOption(Some item) -> matchPattern itemPattern item | Lir.NonePattern, VOption None -> Some Map.empty + | Lir.UnionPattern(typeName, caseName, payloadPattern), VUnion(valueType, valueCase, payload) when typeName = valueType && caseName = valueCase -> + match payloadPattern, payload with None, None -> Some Map.empty | Some pattern, Some value -> matchPattern pattern value | _ -> None | _ -> None let rec private evaluate environment (expression: Lir.Expression) = @@ -202,6 +207,9 @@ module LirInterpreter = | Lir.Interpolate parts -> parts |> List.map (function Lir.TextPart text -> Ok text | Lir.ValuePart item -> evaluate environment item |> Result.map formatValue) |> collect |> Result.map (String.concat "" >> VString) + | Lir.UnionCase(typeName, caseName, payload) -> + payload |> Option.map (fun value -> evaluate environment value |> Result.map Some) |> Option.defaultValue (Ok None) + |> Result.map (fun value -> VUnion(typeName, caseName, value)) and private evaluateMany environment items = items |> List.map (evaluate environment) |> collect diff --git a/tests/FScript.Compiler.Tests/GoBackendTests.fs b/tests/FScript.Compiler.Tests/GoBackendTests.fs index 9cad974..4ffca93 100644 --- a/tests/FScript.Compiler.Tests/GoBackendTests.fs +++ b/tests/FScript.Compiler.Tests/GoBackendTests.fs @@ -19,6 +19,7 @@ type GoBackendTests() = [] [] [] + [ 0\n | Circle r -> r\nradius (Circle 3)", "3")>] member _.``Generated projects run with Go`` (source: string, expected: string) = let input = { Program = Parser.parseProgram source; Externs = [] } let compiled = diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index 7e17760..abdea95 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -90,3 +90,13 @@ type HirTests() = match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(expected), source) | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``Executes declared union construction and matching`` () = + let source = "type Shape = | Point | Circle of int\nlet radius shape =\n match shape with\n | Point -> 0\n | Circle r -> r\nradius (Circle 3)" + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 3L)) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index e724c79..b538c8c 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -47,7 +47,8 @@ type IrArtifactTests() = "raise \"failure\"" "match Some 2 with\n | Some x when x > 0 -> x\n | None -> 0" "([1..3][0], $\"answer {42}\")" - "let value = {| Count = 1 |}\n{| value with Count = 2 |}" ] do + "let value = {| Count = 1 |}\n{| value with Count = 2 |}" + "type Shape = | Point | Circle of int\nmatch Circle 2 with\n | Point -> 0\n | Circle radius -> radius" ] do let bytes = compile source |> _.Lir |> IrArtifact.serialize match IrArtifact.deserialize bytes with | Ok _ -> () From be746229a4acec47f4c693a7a4830d9864732d66 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 17:04:27 +0200 Subject: [PATCH 013/133] feat(compiler): execute deterministic ordered maps --- CHANGELOG.md | 1 + docs/specs/compiler-semantics-baseline.md | 2 ++ src/FScript.Compiler/GoBackend.fs | 14 ++++++++++- src/FScript.Compiler/Hir.fs | 8 +++++++ src/FScript.Compiler/HirLowering.fs | 9 +++++++ src/FScript.Compiler/HirValidation.fs | 2 ++ src/FScript.Compiler/IrArtifact.fs | 15 ++++++++++++ src/FScript.Compiler/Lir.fs | 6 +++++ src/FScript.Compiler/LirInterpreter.fs | 24 +++++++++++++++++++ .../FScript.Compiler.Tests/GoBackendTests.fs | 1 + tests/FScript.Compiler.Tests/HirTests.fs | 10 ++++++++ .../FScript.Compiler.Tests/IrArtifactTests.fs | 3 ++- 12 files changed, 93 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 20d9d65..e3dabb2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added deterministic ordered-map construction, left-precedence spreading, indexing, equality, serialization, and Go runtime support. - Added declared-union constructors, payloads, structural equality, pattern matching, artifact serialization, and Go runtime representation. - Added compiled ranges, safe list indexing, immutable record updates, interpolation, and FScript-style generated-program result formatting. - Added deterministic LIR pattern decision trees with single scrutinee evaluation, ordered guards, bindings, and match-failure diagnostics. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 78ff69e..d21226b 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -16,6 +16,8 @@ The first aggregate compiler slice preserves tuple field order, record field nam Ranges preserve inclusive ascending or descending order. List indexing returns an option and never throws for an out-of-range integer. Record updates allocate a new value, interpolation evaluates embedded expressions from left to right, and generated program output uses FScript value formatting rather than Go's default aggregate formatting. Declared unions retain their type and case identity plus an optional erased payload. Constructors are deterministic compiler declarations, qualified and unqualified case references share one identity, pattern tests check both type and case tags, and equality recursively compares payloads. + +Compiled maps use an ordered entry representation for string and integer keys. Explicit key/value entries replace an earlier value; spread entries retain values already present on their left. Indexing returns an option, and deterministic ordering participates in artifacts, equality, formatting, and generated Go behavior. - Functions are curried. Multi-argument declarations are nested lambdas, application is one argument at a time, and closures capture their lexical environment. Single and mutual recursive function groups are supported. ## Types and values diff --git a/src/FScript.Compiler/GoBackend.fs b/src/FScript.Compiler/GoBackend.fs index ad4993e..6b9dd4e 100644 --- a/src/FScript.Compiler/GoBackend.fs +++ b/src/FScript.Compiler/GoBackend.fs @@ -21,6 +21,7 @@ module GoBackend = | TOption _ -> "fsruntime.Option" | TRecord _ -> "map[string]any" | TUnion _ | TNamed _ -> "fsruntime.Union" + | TMap _ -> "fsruntime.OrderedMap" | value -> invalidArg "value" $"Unsupported Go representation for {Types.typeToString value}." let private identifier prefix id = $"{prefix}{Lir.symbolNumber id}" @@ -57,6 +58,7 @@ module GoBackend = | TRecord _ -> $"fsruntime.Record({code})" | TOption _ -> $"fsruntime.AsOption({code})" | TUnion _ | TNamed _ -> $"fsruntime.AsUnion({code})" + | TMap _ -> $"fsruntime.AsMap({code})" | _ -> code let rec private patternBindings (pattern: Lir.Pattern) = @@ -221,6 +223,16 @@ module GoBackend = match payload with | None -> Ok $"fsruntime.MakeUnion({quotedType}, {quotedCase})" | Some item -> emitExpression environment item |> Result.map (fun code -> $"fsruntime.MakeUnionValue({quotedType}, {quotedCase}, {code})") + | Lir.Map entries -> + entries |> List.map (function + | Lir.KeyValue(key, value) -> + match emitExpression environment key, emitExpression environment value with + | Ok keyCode, Ok valueCode -> Ok $"fsruntime.Pair({keyCode}, {valueCode})" + | Error error, _ | _, Error error -> Error error + | Lir.Spread value -> emitExpression environment value |> Result.map (fun code -> $"fsruntime.SpreadMap({code})")) + |> collect |> Result.map (fun parts -> + let joined = String.concat ", " parts + $"fsruntime.BuildMap([]fsruntime.MapPart{{{joined}}})") | Lir.Operation _ -> Error "Invalid operation arity" and private collect<'a> (results: Result<'a, string> list) : Result<'a list, string> = @@ -254,6 +266,6 @@ module GoBackend = [ { RelativePath = "go.mod"; Content = $"module {moduleName}\n\ngo 1.25\n" } { RelativePath = "main.go"; Content = $"package main\n\nimport (\n\t\"fmt\"\n\t\"{moduleName}/internal/fscriptprogram\"\n\t\"{moduleName}/internal/fsruntime\"\n)\n\nfunc main() {{ fmt.Println(fsruntime.Format(fscriptprogram.Run())) }}\n" } { RelativePath = "internal/fscriptprogram/program.go"; Content = programSource } - { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"strings\"\n)\n\ntype Option struct { HasValue bool; Value any }\ntype Union struct { TypeName string; CaseName string; HasValue bool; Value any }\n\nfunc Int(value any) int64 { return value.(int64) }\nfunc Float(value any) float64 { return value.(float64) }\nfunc Bool(value any) bool { return value.(bool) }\nfunc String(value any) string { return value.(string) }\nfunc List(value any) []any { return value.([]any) }\nfunc Record(value any) map[string]any { return value.(map[string]any) }\nfunc AsOption(value any) Option { return value.(Option) }\nfunc AsUnion(value any) Union { return value.(Union) }\nfunc Some(value any) Option { return Option{HasValue: true, Value: value} }\nfunc None() Option { return Option{} }\nfunc IsSome(value any) bool { return AsOption(value).HasValue }\nfunc OptionValue(value any) any { return AsOption(value).Value }\nfunc MakeUnion(typeName string, caseName string) Union { return Union{TypeName: typeName, CaseName: caseName} }\nfunc MakeUnionValue(typeName string, caseName string, value any) Union { return Union{TypeName: typeName, CaseName: caseName, HasValue: true, Value: value} }\nfunc IsUnion(value any, typeName string, caseName string) bool { item := AsUnion(value); return item.TypeName == typeName && item.CaseName == caseName }\nfunc UnionHasValue(value any) bool { return AsUnion(value).HasValue }\nfunc UnionValue(value any) any { return AsUnion(value).Value }\nfunc HasField(value any, name string) bool { _, ok := Record(value)[name]; return ok }\nfunc Equal(left any, right any) bool { return reflect.DeepEqual(left, right) }\nfunc Cons(head any, tail []any) []any { return append([]any{head}, tail...) }\nfunc Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) }\nfunc Tail(value any) []any { return append([]any{}, List(value)[1:]...) }\nfunc Range(first int64, last int64) []any { step := int64(1); if first > last { step = -1 }; values := []any{}; for current := first; ; current += step { values = append(values, current); if current == last { break } }; return values }\nfunc UpdateRecord(value any, updates map[string]any) map[string]any { result := map[string]any{}; for key, item := range Record(value) { result[key] = item }; for key, item := range updates { result[key] = item }; return result }\nfunc Index(value any, index any) Option { values := List(value); position := Int(index); if position < 0 || position >= int64(len(values)) { return None() }; return Some(values[position]) }\nfunc Format(value any) string { switch item := value.(type) { case bool: if item { return \"true\" }; return \"false\"; case Option: if !item.HasValue { return \"None\" }; return \"Some \" + Format(item.Value); case Union: if !item.HasValue { return item.CaseName }; return item.CaseName + \" \" + Format(item.Value); case []any: parts := make([]string, len(item)); for index, value := range item { parts[index] = Format(value) }; return \"[\" + strings.Join(parts, \";\") + \"]\"; default: return fmt.Sprint(value) } }\nfunc Interpolate(values []any) string { var result strings.Builder; for _, value := range values { result.WriteString(Format(value)) }; return result.String() }\nfunc Fail(value any) { panic(fmt.Sprint(value)) }\n" } ] + { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"sort\"\n\t\"strings\"\n)\n\ntype Option struct { HasValue bool; Value any }\ntype Union struct { TypeName string; CaseName string; HasValue bool; Value any }\ntype MapEntry struct { Key any; Value any }\ntype OrderedMap struct { Entries []MapEntry }\ntype MapPart struct { IsSpread bool; Key any; Value any; Map OrderedMap }\n\nfunc Int(value any) int64 { return value.(int64) }\nfunc Float(value any) float64 { return value.(float64) }\nfunc Bool(value any) bool { return value.(bool) }\nfunc String(value any) string { return value.(string) }\nfunc List(value any) []any { return value.([]any) }\nfunc Record(value any) map[string]any { return value.(map[string]any) }\nfunc AsOption(value any) Option { return value.(Option) }\nfunc AsUnion(value any) Union { return value.(Union) }\nfunc AsMap(value any) OrderedMap { return value.(OrderedMap) }\nfunc Some(value any) Option { return Option{HasValue: true, Value: value} }\nfunc None() Option { return Option{} }\nfunc IsSome(value any) bool { return AsOption(value).HasValue }\nfunc OptionValue(value any) any { return AsOption(value).Value }\nfunc MakeUnion(typeName string, caseName string) Union { return Union{TypeName: typeName, CaseName: caseName} }\nfunc MakeUnionValue(typeName string, caseName string, value any) Union { return Union{TypeName: typeName, CaseName: caseName, HasValue: true, Value: value} }\nfunc IsUnion(value any, typeName string, caseName string) bool { item := AsUnion(value); return item.TypeName == typeName && item.CaseName == caseName }\nfunc UnionHasValue(value any) bool { return AsUnion(value).HasValue }\nfunc UnionValue(value any) any { return AsUnion(value).Value }\nfunc HasField(value any, name string) bool { _, ok := Record(value)[name]; return ok }\nfunc Equal(left any, right any) bool { return reflect.DeepEqual(left, right) }\nfunc Cons(head any, tail []any) []any { return append([]any{head}, tail...) }\nfunc Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) }\nfunc Tail(value any) []any { return append([]any{}, List(value)[1:]...) }\nfunc Range(first int64, last int64) []any { step := int64(1); if first > last { step = -1 }; values := []any{}; for current := first; ; current += step { values = append(values, current); if current == last { break } }; return values }\nfunc UpdateRecord(value any, updates map[string]any) map[string]any { result := map[string]any{}; for key, item := range Record(value) { result[key] = item }; for key, item := range updates { result[key] = item }; return result }\nfunc Pair(key any, value any) MapPart { return MapPart{Key: key, Value: value} }\nfunc SpreadMap(value any) MapPart { return MapPart{IsSpread: true, Map: AsMap(value)} }\nfunc mapSet(entries []MapEntry, key any, value any, replace bool) []MapEntry { for index, entry := range entries { if Equal(entry.Key, key) { if replace { entries[index].Value = value }; return entries } }; return append(entries, MapEntry{Key: key, Value: value}) }\nfunc BuildMap(parts []MapPart) OrderedMap { entries := []MapEntry{}; for _, part := range parts { if part.IsSpread { for _, entry := range part.Map.Entries { entries = mapSet(entries, entry.Key, entry.Value, false) } } else { entries = mapSet(entries, part.Key, part.Value, true) } }; sort.SliceStable(entries, func(left int, right int) bool { return Format(entries[left].Key) < Format(entries[right].Key) }); return OrderedMap{Entries: entries} }\nfunc Index(value any, index any) Option { switch target := value.(type) { case []any: position := Int(index); if position < 0 || position >= int64(len(target)) { return None() }; return Some(target[position]); case OrderedMap: for _, entry := range target.Entries { if Equal(entry.Key, index) { return Some(entry.Value) } }; return None(); default: panic(\"Index access requires a list or map\") } }\nfunc Format(value any) string { switch item := value.(type) { case bool: if item { return \"true\" }; return \"false\"; case Option: if !item.HasValue { return \"None\" }; return \"Some \" + Format(item.Value); case Union: if !item.HasValue { return item.CaseName }; return item.CaseName + \" \" + Format(item.Value); case OrderedMap: parts := make([]string, len(item.Entries)); for index, entry := range item.Entries { parts[index] = \"[\" + Format(entry.Key) + \"] = \" + Format(entry.Value) }; return \"{ \" + strings.Join(parts, \"; \") + \" }\"; case []any: parts := make([]string, len(item)); for index, value := range item { parts[index] = Format(value) }; return \"[\" + strings.Join(parts, \";\") + \"]\"; default: return fmt.Sprint(value) } }\nfunc Interpolate(values []any) string { var result strings.Builder; for _, value := range values { result.WriteString(Format(value)) }; return result.String() }\nfunc Fail(value any) { panic(fmt.Sprint(value)) }\n" } ] with ex -> Error [ { Code = "FSB4002"; Stage = GoGeneration; Message = ex.Message; Span = None } ] diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs index 0db4c34..9f3ccab 100644 --- a/src/FScript.Compiler/Hir.fs +++ b/src/FScript.Compiler/Hir.fs @@ -58,6 +58,11 @@ module Hir = | IndexGet of Expression * Expression | Interpolate of InterpolationPart list | UnionCase of string * string * Expression option + | Map of MapEntry list + + and MapEntry = + | KeyValue of Expression * Expression + | Spread of Expression and InterpolationPart = | TextPart of string @@ -167,6 +172,9 @@ module Hir = | UnionCase(typeName, caseName, payload) -> lines.Add(typed $"union {typeName}.{caseName}") payload |> Option.iter (expression (indent + 2)) + | Map entries -> + lines.Add(typed "map") + entries |> List.iter (function KeyValue(key, value) -> expression (indent + 2) key; expression (indent + 2) value | Spread value -> expression (indent + 2) value) for declaration in program.Declarations do lines.Add(sprintf "decl %%%d %s: %s" (symbolNumber declaration.Id) declaration.Name (formatType (let (Forall(_, value)) = declaration.Scheme in value))) diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index 9cd9d5c..b2251a9 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -235,6 +235,15 @@ module HirLowering = | IPText text -> Ok(Hir.TextPart text) | IPExpr item -> lowerExpression state environment (expressionType state item (Types.freshVar())) item |> Result.map Hir.ValuePart) |> collectResults |> Result.map (Hir.Interpolate >> make) + | EMap(entries, _) -> + let keyType, valueType = match inferred with TMap(key, value) -> key, value | _ -> Types.freshVar(), Types.freshVar() + entries |> List.map (function + | MEKeyValue(key, value) -> + match lowerExpression state environment keyType key, lowerExpression state environment valueType value with + | Ok loweredKey, Ok loweredValue -> Ok(Hir.KeyValue(loweredKey, loweredValue)) + | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e + | MESpread value -> lowerExpression state environment inferred value |> Result.map Hir.Spread) + |> collectResults |> Result.map (Hir.Map >> make) | _ -> Error [ Diagnostic.hir "FSH1099" $"HIR lowering does not yet support {expression.GetType().Name}." (Some span) ] let lower (resolved: TypeInfer.ResolvedProgram) : Result = diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index a14fb26..e2aee56 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -46,6 +46,8 @@ module HirValidation = | Hir.RecordUpdate(target, fields) -> Set.union (references target) (fields |> List.map (snd >> references) |> Set.unionMany) | Hir.Interpolate parts -> parts |> List.choose (function Hir.TextPart _ -> None | Hir.ValuePart item -> Some(references item)) |> Set.unionMany | Hir.UnionCase(_, _, payload) -> payload |> Option.map references |> Option.defaultValue Set.empty + | Hir.Map entries -> + entries |> List.map (function Hir.KeyValue(key, value) -> Set.union (references key) (references value) | Hir.Spread value -> references value) |> Set.unionMany let declared = declarationIds |> Set.ofList let unresolved = [ yield! program.Declarations |> List.collect (fun value -> references value.Value |> Set.toList) diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index 821a4e8..68404c3 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -175,6 +175,15 @@ module IrArtifact = | Lir.UnionCase(typeName, caseName, payload) -> writer.WriteString("kind", "union-case"); writer.WriteString("typeName", typeName); writer.WriteString("caseName", caseName) match payload with Some item -> writer.WritePropertyName("payload"); writeExpression writer item | None -> writer.WriteNull("payload") + | Lir.Map entries -> + writer.WriteString("kind", "map"); writer.WriteStartArray("entries") + for entry in entries do + writer.WriteStartObject() + match entry with + | Lir.KeyValue(key, value) -> writer.WriteString("kind", "key-value"); writer.WritePropertyName("key"); writeExpression writer key; writer.WritePropertyName("value"); writeExpression writer value + | Lir.Spread value -> writer.WriteString("kind", "spread"); writer.WritePropertyName("value"); writeExpression writer value + writer.WriteEndObject() + writer.WriteEndArray() writer.WriteEndObject() let serialize (program: Lir.Program) = @@ -300,6 +309,12 @@ module IrArtifact = | "union-case" -> let payload = property "payload" element Lir.UnionCase(stringProperty "typeName" element, stringProperty "caseName" element, if payload.ValueKind = JsonValueKind.Null then None else Some(readExpression payload)) + | "map" -> + Lir.Map((property "entries" element).EnumerateArray() |> Seq.map (fun item -> + match stringProperty "kind" item with + | "key-value" -> Lir.KeyValue(readExpression (property "key" item), readExpression (property "value" item)) + | "spread" -> Lir.Spread(readExpression (property "value" item)) + | kind -> invalidOp $"Unknown map entry '{kind}'.") |> Seq.toList) | kind -> invalidOp $"Unknown LIR expression kind '{kind}'." { Type = expressionType; Span = span; Node = node } diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index 0b77d5e..c6a06ac 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -46,6 +46,11 @@ module Lir = | IndexGet of Expression * Expression | Interpolate of InterpolationPart list | UnionCase of string * string * Expression option + | Map of MapEntry list + + and MapEntry = + | KeyValue of Expression * Expression + | Spread of Expression and InterpolationPart = | TextPart of string @@ -152,6 +157,7 @@ module LirLowering = | Hir.IndexGet(target, index) -> Lir.IndexGet(expression target, expression index) | Hir.Interpolate parts -> Lir.Interpolate(parts |> List.map (function Hir.TextPart text -> Lir.TextPart text | Hir.ValuePart item -> Lir.ValuePart(expression item))) | Hir.UnionCase(typeName, caseName, payload) -> Lir.UnionCase(typeName, caseName, Option.map expression payload) + | Hir.Map entries -> Lir.Map(entries |> List.map (function Hir.KeyValue(key, item) -> Lir.KeyValue(expression key, expression item) | Hir.Spread item -> Lir.Spread(expression item))) { Type = value.Type; Span = value.Span; Node = node } let lower (program: Hir.Program) : Lir.Program = diff --git a/src/FScript.Compiler/LirInterpreter.fs b/src/FScript.Compiler/LirInterpreter.fs index c3457d6..2195949 100644 --- a/src/FScript.Compiler/LirInterpreter.fs +++ b/src/FScript.Compiler/LirInterpreter.fs @@ -17,6 +17,7 @@ type LirValue = | VRecord of Map | VOption of LirValue option | VUnion of string * string * LirValue option + | VMap of (LirValue * LirValue) list | VClosure of Lir.SymbolId * string * Lir.Expression * Map ref type ExecutionResult = { Value: LirValue } @@ -38,6 +39,7 @@ module LirInterpreter = | VOption None, VOption None -> true | VOption(Some a), VOption(Some b) -> equal a b | VUnion(aType, aCase, aValue), VUnion(bType, bCase, bValue) -> aType = bType && aCase = bCase && equal (VOption aValue) (VOption bValue) + | VMap a, VMap b -> a.Length = b.Length && List.forall2 (fun (ak, av) (bk, bv) -> equal ak bk && equal av bv) a b | _ -> false let rec private formatValue = function @@ -50,6 +52,8 @@ module LirInterpreter = "{ " + rendered + " }" | VOption None -> "None" | VOption(Some value) -> $"Some {formatValue value}" | VUnion(_, caseName, None) -> caseName | VUnion(_, caseName, Some value) -> $"{caseName} {formatValue value}" + | VMap entries -> + entries |> List.map (fun (key, value) -> $"[{formatValue key}] = {formatValue value}") |> String.concat "; " |> fun value -> "{ " + value + " }" | VClosure _ -> "" let private fail code span message = @@ -202,6 +206,7 @@ module LirInterpreter = | Ok(VList values), Ok(VInt position) -> if position < 0L || position >= int64 values.Length then Ok(VOption None) else Ok(VOption(Some values.[int position])) + | Ok(VMap entries), Ok key -> Ok(VOption(entries |> List.tryPick (fun (itemKey, value) -> if equal itemKey key then Some value else None))) | Ok _, Ok _ -> fail "FSI3016" expression.Span "Index access requires a list and integer index." | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e | Lir.Interpolate parts -> @@ -210,6 +215,25 @@ module LirInterpreter = | Lir.UnionCase(typeName, caseName, payload) -> payload |> Option.map (fun value -> evaluate environment value |> Result.map Some) |> Option.defaultValue (Ok None) |> Result.map (fun value -> VUnion(typeName, caseName, value)) + | Lir.Map entries -> + let add key value values = + let without = values |> List.filter (fun (existing, _) -> not (equal existing key)) + (key, value) :: without |> List.sortBy (fst >> formatValue) + let rec build values remaining = + match remaining with + | [] -> Ok(VMap values) + | Lir.KeyValue(key, item) :: rest -> + match evaluate environment key, evaluate environment item with + | Ok keyValue, Ok itemValue -> build (add keyValue itemValue values) rest + | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e + | Lir.Spread item :: rest -> + match evaluate environment item with + | Ok(VMap spread) -> + let merged = spread |> List.fold (fun state (key, value) -> if state |> List.exists (fst >> equal key) then state else add key value state) values + build merged rest + | Ok _ -> fail "FSI3017" expression.Span "Map spread requires a map." + | Error errors -> Error errors + build [] entries and private evaluateMany environment items = items |> List.map (evaluate environment) |> collect diff --git a/tests/FScript.Compiler.Tests/GoBackendTests.fs b/tests/FScript.Compiler.Tests/GoBackendTests.fs index 4ffca93..1bc4619 100644 --- a/tests/FScript.Compiler.Tests/GoBackendTests.fs +++ b/tests/FScript.Compiler.Tests/GoBackendTests.fs @@ -20,6 +20,7 @@ type GoBackendTests() = [] [] [ 0\n | Circle r -> r\nradius (Circle 3)", "3")>] + [] member _.``Generated projects run with Go`` (source: string, expected: string) = let input = { Program = Parser.parseProgram source; Externs = [] } let compiled = diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index abdea95..ada875e 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -100,3 +100,13 @@ type HirTests() = match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 3L)) | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``Executes ordered map construction spread and indexing`` () = + let source = "let tail = { [\"a\"] = 2; [\"b\"] = 3 }\nlet values = { [\"a\"] = 1; ..tail }\nvalues[\"b\"]" + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VOption(Some(LirValue.VInt 3L)))) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index b538c8c..75d594e 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -48,7 +48,8 @@ type IrArtifactTests() = "match Some 2 with\n | Some x when x > 0 -> x\n | None -> 0" "([1..3][0], $\"answer {42}\")" "let value = {| Count = 1 |}\n{| value with Count = 2 |}" - "type Shape = | Point | Circle of int\nmatch Circle 2 with\n | Point -> 0\n | Circle radius -> radius" ] do + "type Shape = | Point | Circle of int\nmatch Circle 2 with\n | Point -> 0\n | Circle radius -> radius" + "{ [\"answer\"] = 42 }[\"answer\"]" ] do let bytes = compile source |> _.Lir |> IrArtifact.serialize match IrArtifact.deserialize bytes with | Ok _ -> () From 2b9906b2248a9bf9e9b4ce568d13ab5ad42d1069 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 17:06:27 +0200 Subject: [PATCH 014/133] feat(compiler): initialize recursive closure groups --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 ++ src/FScript.Compiler/GoBackend.fs | 6 +++-- src/FScript.Compiler/HirLowering.fs | 13 +++++++++++ src/FScript.Compiler/LirInterpreter.fs | 23 ++++++++++++++----- .../FScript.Compiler.Tests/GoBackendTests.fs | 1 + tests/FScript.Compiler.Tests/HirTests.fs | 10 ++++++++ 7 files changed, 48 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e3dabb2..0703e47 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added atomic top-level mutual-recursive closure initialization in the LIR interpreter and generated Go projects. - Added deterministic ordered-map construction, left-precedence spreading, indexing, equality, serialization, and Go runtime support. - Added declared-union constructors, payloads, structural equality, pattern matching, artifact serialization, and Go runtime representation. - Added compiled ranges, safe list indexing, immutable record updates, interpolation, and FScript-style generated-program result formatting. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index ce15061..19e1676 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -24,6 +24,8 @@ Lexical `let` is retained as a structured LIR node. Its symbol is visible only i Matches lower to a structured decision tree whose target is evaluated once. Cases remain ordered; each pattern test creates artifact-local bindings, then its guard is evaluated, and only then is its body selected. A failed guard continues at the next case. Exhaustion is the explicit FScript match-failure path. Literal, list, tuple, record, and option patterns use the same tree in serialized artifacts, the LIR interpreter, and generated Go. +Top-level mutual-recursive declarations are initialized as an atomic closure group. The interpreter creates every closure against the preceding environment, extends that environment with the complete group, and updates all captures together. Generated Go declares typed slots for all declarations and assigns them in source order from `init`, allowing closures in one recursive group to refer to every peer. + Serialized `.fsir` files contain one closed resolved program plus explicit foreign linkage declarations. JSON format versions are internal and may be rejected rather than upgraded. ## Representation policy diff --git a/src/FScript.Compiler/GoBackend.fs b/src/FScript.Compiler/GoBackend.fs index 6b9dd4e..14d01cf 100644 --- a/src/FScript.Compiler/GoBackend.fs +++ b/src/FScript.Compiler/GoBackend.fs @@ -245,10 +245,12 @@ module GoBackend = program.Declarations |> List.fold (fun env declaration -> env |> Map.add declaration.Id (identifier "value" declaration.Id)) Map.empty let declarations = StringBuilder() + let initializers = StringBuilder() let mutable error = None for declaration in program.Declarations do + declarations.AppendLine($"var {names.[declaration.Id]} {goType declaration.Type}") |> ignore match emitExpression names declaration.Value with - | Ok code -> declarations.AppendLine($"var {names.[declaration.Id]} = {code}") |> ignore + | Ok code -> initializers.AppendLine($"\t{names.[declaration.Id]} = {code}") |> ignore | Error message -> error <- Some(message, declaration.Span) let entry = match program.Entry with @@ -261,7 +263,7 @@ module GoBackend = Error [ { Code = "FSB4001"; Stage = GoGeneration; Message = message; Span = None } ] | None, Ok entryCode -> let moduleName = if String.IsNullOrWhiteSpace options.ModuleName then "fscript.generated/program" else options.ModuleName - let programSource = $"package fscriptprogram\n\nimport \"{moduleName}/internal/fsruntime\"\n\n{declarations}\nfunc Run() any {{ return {entryCode} }}\n" + let programSource = $"package fscriptprogram\n\nimport \"{moduleName}/internal/fsruntime\"\n\n{declarations}\nfunc init() {{\n{initializers}}}\n\nfunc Run() any {{ return {entryCode} }}\n" Ok [ { RelativePath = "go.mod"; Content = $"module {moduleName}\n\ngo 1.25\n" } { RelativePath = "main.go"; Content = $"package main\n\nimport (\n\t\"fmt\"\n\t\"{moduleName}/internal/fscriptprogram\"\n\t\"{moduleName}/internal/fsruntime\"\n)\n\nfunc main() {{ fmt.Println(fsruntime.Format(fscriptprogram.Run())) }}\n" } diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index b2251a9..6702550 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -322,6 +322,19 @@ module HirLowering = match lowerExpression state topLevel expression.Type expression.Expr with | Ok value -> entry <- Some value | Error diagnostics -> errors <- errors @ diagnostics + | TypeInfer.TSLetRecGroup(bindings, isExported, groupSpan) -> + for name, expression, inferred, span in bindings do + match lowerExpression state topLevel inferred expression with + | Ok value -> + declarations <- declarations @ + [ ({ Id = Map.find name topLevel + Name = name + Scheme = Types.generalize Map.empty inferred + IsRecursive = true + IsExported = isExported + Span = span + Value = value }: Hir.Declaration) ] + | Error diagnostics -> errors <- errors @ diagnostics | TypeInfer.TSType _ -> () | _ -> errors <- errors @ [ Diagnostic.hir "FSH1098" "Top-level declaration form is not yet supported by HIR lowering." None ] if errors.IsEmpty then Ok { Declarations = declarations; Entry = entry } diff --git a/src/FScript.Compiler/LirInterpreter.fs b/src/FScript.Compiler/LirInterpreter.fs index 2195949..be28c96 100644 --- a/src/FScript.Compiler/LirInterpreter.fs +++ b/src/FScript.Compiler/LirInterpreter.fs @@ -244,15 +244,26 @@ module LirInterpreter = let execute (_context: InterpreterContext) (program: Lir.Program) = let environment = ref Map.empty let mutable failure = None - for declaration in program.Declarations do - if failure.IsNone then + let rec initialize (declarations: Lir.Declaration list) = + match declarations with + | [] -> () + | declaration :: rest when not declaration.IsRecursive -> match evaluate environment.Value declaration.Value with | Error errors -> failure <- Some errors - | Ok(VClosure(id, name, body, captured) as closure) -> - let next = environment.Value |> Map.add declaration.Id closure - if declaration.IsRecursive then captured.Value <- next + | Ok value -> environment.Value <- environment.Value |> Map.add declaration.Id value; initialize rest + | _ -> + let group = declarations |> List.takeWhile _.IsRecursive + let remainder = declarations |> List.skip group.Length + let closures = group |> List.map (fun declaration -> declaration, evaluate environment.Value declaration.Value) + let errors = closures |> List.collect (fun (_, result) -> match result with Error values -> values | Ok _ -> []) + if not errors.IsEmpty then failure <- Some errors + else + let next = closures |> List.fold (fun env (declaration, result) -> match result with Ok closure -> Map.add declaration.Id closure env | Error _ -> env) environment.Value + for _, result in closures do + match result with Ok(VClosure(_, _, _, captured)) -> captured.Value <- next | _ -> () environment.Value <- next - | Ok value -> environment.Value <- environment.Value |> Map.add declaration.Id value + initialize remainder + initialize program.Declarations match failure, program.Entry with | Some errors, _ -> Error errors | None, Some entry -> evaluate environment.Value entry |> Result.map (fun value -> { Value = value }) diff --git a/tests/FScript.Compiler.Tests/GoBackendTests.fs b/tests/FScript.Compiler.Tests/GoBackendTests.fs index 1bc4619..24ce3e2 100644 --- a/tests/FScript.Compiler.Tests/GoBackendTests.fs +++ b/tests/FScript.Compiler.Tests/GoBackendTests.fs @@ -21,6 +21,7 @@ type GoBackendTests() = [] [ 0\n | Circle r -> r\nradius (Circle 3)", "3")>] [] + [] member _.``Generated projects run with Go`` (source: string, expected: string) = let input = { Program = Parser.parseProgram source; Externs = [] } let compiled = diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index ada875e..aa10544 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -110,3 +110,13 @@ type HirTests() = match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VOption(Some(LirValue.VInt 3L)))) | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``Initializes mutual recursive groups atomically`` () = + let source = "let rec even n =\n if n = 0 then true else odd (n - 1)\nand odd n =\n if n = 0 then false else even (n - 1)\n\neven 10" + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VBool true)) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) From f3cf99c10275851c3845c00b2a53c6f876102723 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 17:11:02 +0200 Subject: [PATCH 015/133] feat(compiler): add portable runtime intrinsics --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 + src/FScript.Compiler/GoBackend.fs | 16 ++++++- src/FScript.Compiler/Hir.fs | 2 + src/FScript.Compiler/HirLowering.fs | 16 +++++-- src/FScript.Compiler/HirValidation.fs | 1 + src/FScript.Compiler/IrArtifact.fs | 2 + src/FScript.Compiler/Lir.fs | 2 + src/FScript.Compiler/LirInterpreter.fs | 45 +++++++++++++++++++ .../FScript.Compiler.Tests/GoBackendTests.fs | 3 ++ tests/FScript.Compiler.Tests/HirTests.fs | 12 +++++ 11 files changed, 98 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0703e47..b0baf72 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added explicit portable intrinsic declarations and LIR/Go execution for scalar parsing and formatting plus core string, list, option, and map queries. - Added atomic top-level mutual-recursive closure initialization in the LIR interpreter and generated Go projects. - Added deterministic ordered-map construction, left-precedence spreading, indexing, equality, serialization, and Go runtime support. - Added declared-union constructors, payloads, structural equality, pattern matching, artifact serialization, and Go runtime representation. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 19e1676..1561c26 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -26,6 +26,8 @@ Matches lower to a structured decision tree whose target is evaluated once. Case Top-level mutual-recursive declarations are initialized as an atomic closure group. The interpreter creates every closure against the preceding environment, extends that environment with the complete group, and updates all captures together. Generated Go declares typed slots for all declarations and assigns them in source order from `init`, allowing closures in one recursive group to refer to every peer. +Language builtins are explicit `Intrinsic` declarations rather than magic unresolved references. Consumers accumulate their curried arguments according to the catalog signature and dispatch only at full application. The initial portable implementations cover scalar parsing/formatting, strings, simple list and option operations, and map queries; a reachable catalog entry without an implementation receives a stage-specific unsupported diagnostic or runtime failure. + Serialized `.fsir` files contain one closed resolved program plus explicit foreign linkage declarations. JSON format versions are internal and may be rejected rather than upgraded. ## Representation policy diff --git a/src/FScript.Compiler/GoBackend.fs b/src/FScript.Compiler/GoBackend.fs index 14d01cf..c06c789 100644 --- a/src/FScript.Compiler/GoBackend.fs +++ b/src/FScript.Compiler/GoBackend.fs @@ -50,6 +50,7 @@ module GoBackend = let private cast valueType code = match valueType with + | TUnit -> $"fsruntime.Unit({code})" | TInt -> $"fsruntime.Int({code})" | TFloat -> $"fsruntime.Float({code})" | TBool -> $"fsruntime.Bool({code})" @@ -110,6 +111,18 @@ module GoBackend = let test, assignments = emitPattern $"fsruntime.UnionValue({valueCode})" item $"fsruntime.IsUnion({valueCode}, {quotedType}, {quotedCase}) && fsruntime.UnionHasValue({valueCode}) && ({test})", assignments + let private emitIntrinsic name intrinsicType = + let quoted = System.Text.Json.JsonSerializer.Serialize(name) + let rec build index arguments = function + | TFun(argumentType, resultType) -> + let argument = $"intrinsicArg{index}" + let body = build (index + 1) (arguments @ [ argument ]) resultType + $"func({argument} {goType argumentType}) {goType resultType} {{ return {body} }}" + | resultType -> + let values = String.concat ", " arguments + cast resultType $"fsruntime.Intrinsic({quoted}, []any{{{values}}})" + build 0 [] intrinsicType + let rec private emitExpression environment (expression: Lir.Expression) = match expression.Node with | Lir.Constant value -> Ok(constant value) @@ -233,6 +246,7 @@ module GoBackend = |> collect |> Result.map (fun parts -> let joined = String.concat ", " parts $"fsruntime.BuildMap([]fsruntime.MapPart{{{joined}}})") + | Lir.Intrinsic name -> Ok(emitIntrinsic name expression.Type) | Lir.Operation _ -> Error "Invalid operation arity" and private collect<'a> (results: Result<'a, string> list) : Result<'a list, string> = @@ -268,6 +282,6 @@ module GoBackend = [ { RelativePath = "go.mod"; Content = $"module {moduleName}\n\ngo 1.25\n" } { RelativePath = "main.go"; Content = $"package main\n\nimport (\n\t\"fmt\"\n\t\"{moduleName}/internal/fscriptprogram\"\n\t\"{moduleName}/internal/fsruntime\"\n)\n\nfunc main() {{ fmt.Println(fsruntime.Format(fscriptprogram.Run())) }}\n" } { RelativePath = "internal/fscriptprogram/program.go"; Content = programSource } - { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"sort\"\n\t\"strings\"\n)\n\ntype Option struct { HasValue bool; Value any }\ntype Union struct { TypeName string; CaseName string; HasValue bool; Value any }\ntype MapEntry struct { Key any; Value any }\ntype OrderedMap struct { Entries []MapEntry }\ntype MapPart struct { IsSpread bool; Key any; Value any; Map OrderedMap }\n\nfunc Int(value any) int64 { return value.(int64) }\nfunc Float(value any) float64 { return value.(float64) }\nfunc Bool(value any) bool { return value.(bool) }\nfunc String(value any) string { return value.(string) }\nfunc List(value any) []any { return value.([]any) }\nfunc Record(value any) map[string]any { return value.(map[string]any) }\nfunc AsOption(value any) Option { return value.(Option) }\nfunc AsUnion(value any) Union { return value.(Union) }\nfunc AsMap(value any) OrderedMap { return value.(OrderedMap) }\nfunc Some(value any) Option { return Option{HasValue: true, Value: value} }\nfunc None() Option { return Option{} }\nfunc IsSome(value any) bool { return AsOption(value).HasValue }\nfunc OptionValue(value any) any { return AsOption(value).Value }\nfunc MakeUnion(typeName string, caseName string) Union { return Union{TypeName: typeName, CaseName: caseName} }\nfunc MakeUnionValue(typeName string, caseName string, value any) Union { return Union{TypeName: typeName, CaseName: caseName, HasValue: true, Value: value} }\nfunc IsUnion(value any, typeName string, caseName string) bool { item := AsUnion(value); return item.TypeName == typeName && item.CaseName == caseName }\nfunc UnionHasValue(value any) bool { return AsUnion(value).HasValue }\nfunc UnionValue(value any) any { return AsUnion(value).Value }\nfunc HasField(value any, name string) bool { _, ok := Record(value)[name]; return ok }\nfunc Equal(left any, right any) bool { return reflect.DeepEqual(left, right) }\nfunc Cons(head any, tail []any) []any { return append([]any{head}, tail...) }\nfunc Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) }\nfunc Tail(value any) []any { return append([]any{}, List(value)[1:]...) }\nfunc Range(first int64, last int64) []any { step := int64(1); if first > last { step = -1 }; values := []any{}; for current := first; ; current += step { values = append(values, current); if current == last { break } }; return values }\nfunc UpdateRecord(value any, updates map[string]any) map[string]any { result := map[string]any{}; for key, item := range Record(value) { result[key] = item }; for key, item := range updates { result[key] = item }; return result }\nfunc Pair(key any, value any) MapPart { return MapPart{Key: key, Value: value} }\nfunc SpreadMap(value any) MapPart { return MapPart{IsSpread: true, Map: AsMap(value)} }\nfunc mapSet(entries []MapEntry, key any, value any, replace bool) []MapEntry { for index, entry := range entries { if Equal(entry.Key, key) { if replace { entries[index].Value = value }; return entries } }; return append(entries, MapEntry{Key: key, Value: value}) }\nfunc BuildMap(parts []MapPart) OrderedMap { entries := []MapEntry{}; for _, part := range parts { if part.IsSpread { for _, entry := range part.Map.Entries { entries = mapSet(entries, entry.Key, entry.Value, false) } } else { entries = mapSet(entries, part.Key, part.Value, true) } }; sort.SliceStable(entries, func(left int, right int) bool { return Format(entries[left].Key) < Format(entries[right].Key) }); return OrderedMap{Entries: entries} }\nfunc Index(value any, index any) Option { switch target := value.(type) { case []any: position := Int(index); if position < 0 || position >= int64(len(target)) { return None() }; return Some(target[position]); case OrderedMap: for _, entry := range target.Entries { if Equal(entry.Key, index) { return Some(entry.Value) } }; return None(); default: panic(\"Index access requires a list or map\") } }\nfunc Format(value any) string { switch item := value.(type) { case bool: if item { return \"true\" }; return \"false\"; case Option: if !item.HasValue { return \"None\" }; return \"Some \" + Format(item.Value); case Union: if !item.HasValue { return item.CaseName }; return item.CaseName + \" \" + Format(item.Value); case OrderedMap: parts := make([]string, len(item.Entries)); for index, entry := range item.Entries { parts[index] = \"[\" + Format(entry.Key) + \"] = \" + Format(entry.Value) }; return \"{ \" + strings.Join(parts, \"; \") + \" }\"; case []any: parts := make([]string, len(item)); for index, value := range item { parts[index] = Format(value) }; return \"[\" + strings.Join(parts, \";\") + \"]\"; default: return fmt.Sprint(value) } }\nfunc Interpolate(values []any) string { var result strings.Builder; for _, value := range values { result.WriteString(Format(value)) }; return result.String() }\nfunc Fail(value any) { panic(fmt.Sprint(value)) }\n" } ] + { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"sort\"\n\t\"strconv\"\n\t\"strings\"\n)\n\ntype Option struct { HasValue bool; Value any }\ntype Union struct { TypeName string; CaseName string; HasValue bool; Value any }\ntype MapEntry struct { Key any; Value any }\ntype OrderedMap struct { Entries []MapEntry }\ntype MapPart struct { IsSpread bool; Key any; Value any; Map OrderedMap }\n\nfunc Unit(value any) struct{} { return struct{}{} }\nfunc Int(value any) int64 { return value.(int64) }\nfunc Float(value any) float64 { return value.(float64) }\nfunc Bool(value any) bool { return value.(bool) }\nfunc String(value any) string { return value.(string) }\nfunc List(value any) []any { return value.([]any) }\nfunc Record(value any) map[string]any { return value.(map[string]any) }\nfunc AsOption(value any) Option { return value.(Option) }\nfunc AsUnion(value any) Union { return value.(Union) }\nfunc AsMap(value any) OrderedMap { return value.(OrderedMap) }\nfunc Some(value any) Option { return Option{HasValue: true, Value: value} }\nfunc None() Option { return Option{} }\nfunc IsSome(value any) bool { return AsOption(value).HasValue }\nfunc OptionValue(value any) any { return AsOption(value).Value }\nfunc MakeUnion(typeName string, caseName string) Union { return Union{TypeName: typeName, CaseName: caseName} }\nfunc MakeUnionValue(typeName string, caseName string, value any) Union { return Union{TypeName: typeName, CaseName: caseName, HasValue: true, Value: value} }\nfunc IsUnion(value any, typeName string, caseName string) bool { item := AsUnion(value); return item.TypeName == typeName && item.CaseName == caseName }\nfunc UnionHasValue(value any) bool { return AsUnion(value).HasValue }\nfunc UnionValue(value any) any { return AsUnion(value).Value }\nfunc HasField(value any, name string) bool { _, ok := Record(value)[name]; return ok }\nfunc Equal(left any, right any) bool { return reflect.DeepEqual(left, right) }\nfunc Cons(head any, tail []any) []any { return append([]any{head}, tail...) }\nfunc Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) }\nfunc Tail(value any) []any { return append([]any{}, List(value)[1:]...) }\nfunc Range(first int64, last int64) []any { step := int64(1); if first > last { step = -1 }; values := []any{}; for current := first; ; current += step { values = append(values, current); if current == last { break } }; return values }\nfunc UpdateRecord(value any, updates map[string]any) map[string]any { result := map[string]any{}; for key, item := range Record(value) { result[key] = item }; for key, item := range updates { result[key] = item }; return result }\nfunc Pair(key any, value any) MapPart { return MapPart{Key: key, Value: value} }\nfunc SpreadMap(value any) MapPart { return MapPart{IsSpread: true, Map: AsMap(value)} }\nfunc mapSet(entries []MapEntry, key any, value any, replace bool) []MapEntry { for index, entry := range entries { if Equal(entry.Key, key) { if replace { entries[index].Value = value }; return entries } }; return append(entries, MapEntry{Key: key, Value: value}) }\nfunc BuildMap(parts []MapPart) OrderedMap { entries := []MapEntry{}; for _, part := range parts { if part.IsSpread { for _, entry := range part.Map.Entries { entries = mapSet(entries, entry.Key, entry.Value, false) } } else { entries = mapSet(entries, part.Key, part.Value, true) } }; sort.SliceStable(entries, func(left int, right int) bool { return Format(entries[left].Key) < Format(entries[right].Key) }); return OrderedMap{Entries: entries} }\nfunc Index(value any, index any) Option { switch target := value.(type) { case []any: position := Int(index); if position < 0 || position >= int64(len(target)) { return None() }; return Some(target[position]); case OrderedMap: for _, entry := range target.Entries { if Equal(entry.Key, index) { return Some(entry.Value) } }; return None(); default: panic(\"Index access requires a list or map\") } }\nfunc Intrinsic(name string, args []any) any { switch name { case \"ignore\": return struct{}{}; case \"List.empty\": return []any{}; case \"Map.empty\": return OrderedMap{}; case \"Int.tryParse\": value, error := strconv.ParseInt(String(args[0]), 10, 64); if error != nil { return None() }; return Some(value); case \"Float.tryParse\": value, error := strconv.ParseFloat(String(args[0]), 64); if error != nil { return None() }; return Some(value); case \"Bool.tryParse\": value, error := strconv.ParseBool(String(args[0])); if error != nil { return None() }; return Some(value); case \"Int.toString\", \"Float.toString\", \"Bool.toString\": return Format(args[0]); case \"String.replace\": return strings.ReplaceAll(String(args[2]), String(args[0]), String(args[1])); case \"String.indexOf\": index := strings.Index(String(args[1]), String(args[0])); if index < 0 { return None() }; return Some(int64(index)); case \"String.toLower\": return strings.ToLower(String(args[0])); case \"String.toUpper\": return strings.ToUpper(String(args[0])); case \"String.substring\": start, length, value := Int(args[0]), Int(args[1]), String(args[2]); if start < 0 || length < 0 || start + length > int64(len(value)) { return None() }; return Some(value[start:start+length]); case \"String.concat\": values := List(args[1]); parts := make([]string, len(values)); for index, value := range values { parts[index] = String(value) }; return strings.Join(parts, String(args[0])); case \"String.split\": values := strings.Split(String(args[1]), String(args[0])); result := make([]any, len(values)); for index, value := range values { result[index] = value }; return result; case \"String.endsWith\": return strings.HasSuffix(String(args[1]), String(args[0])); case \"List.length\": return int64(len(List(args[0]))); case \"List.rev\": values := List(args[0]); result := make([]any, len(values)); for index, value := range values { result[len(values)-index-1] = value }; return result; case \"List.tryHead\": values := List(args[0]); if len(values) == 0 { return None() }; return Some(values[0]); case \"List.tail\": return Tail(args[0]); case \"List.append\": return Append(List(args[0]), List(args[1])); case \"List.contains\": for _, value := range List(args[1]) { if Equal(args[0], value) { return true } }; return false; case \"Option.isNone\": return !IsSome(args[0]); case \"Option.isSome\": return IsSome(args[0]); case \"Option.defaultValue\": option := AsOption(args[1]); if option.HasValue { return option.Value }; return args[0]; case \"Map.count\": return int64(len(AsMap(args[0]).Entries)); case \"Map.containsKey\": return IsSome(Index(args[1], args[0])); case \"Map.tryGet\": return Index(args[1], args[0]); default: panic(\"Unsupported FScript intrinsic: \" + name) } }\nfunc Format(value any) string { switch item := value.(type) { case bool: if item { return \"true\" }; return \"false\"; case Option: if !item.HasValue { return \"None\" }; return \"Some \" + Format(item.Value); case Union: if !item.HasValue { return item.CaseName }; return item.CaseName + \" \" + Format(item.Value); case OrderedMap: parts := make([]string, len(item.Entries)); for index, entry := range item.Entries { parts[index] = \"[\" + Format(entry.Key) + \"] = \" + Format(entry.Value) }; return \"{ \" + strings.Join(parts, \"; \") + \" }\"; case []any: parts := make([]string, len(item)); for index, value := range item { parts[index] = Format(value) }; return \"[\" + strings.Join(parts, \";\") + \"]\"; default: return fmt.Sprint(value) } }\nfunc Interpolate(values []any) string { var result strings.Builder; for _, value := range values { result.WriteString(Format(value)) }; return result.String() }\nfunc Fail(value any) { panic(fmt.Sprint(value)) }\n" } ] with ex -> Error [ { Code = "FSB4002"; Stage = GoGeneration; Message = ex.Message; Span = None } ] diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs index 9f3ccab..c42215a 100644 --- a/src/FScript.Compiler/Hir.fs +++ b/src/FScript.Compiler/Hir.fs @@ -59,6 +59,7 @@ module Hir = | Interpolate of InterpolationPart list | UnionCase of string * string * Expression option | Map of MapEntry list + | Intrinsic of string and MapEntry = | KeyValue of Expression * Expression @@ -175,6 +176,7 @@ module Hir = | Map entries -> lines.Add(typed "map") entries |> List.iter (function KeyValue(key, value) -> expression (indent + 2) key; expression (indent + 2) value | Spread value -> expression (indent + 2) value) + | Intrinsic name -> lines.Add(typed $"intrinsic {name}") for declaration in program.Declarations do lines.Add(sprintf "decl %%%d %s: %s" (symbolNumber declaration.Id) declaration.Name (formatType (let (Forall(_, value)) = declaration.Scheme in value))) diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index 6702550..32c8cf0 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -178,7 +178,12 @@ module HirLowering = fields |> List.map (fun (name, item) -> lowerExpression state environment (fieldTypes |> Map.tryFind name |> Option.defaultValue (Types.freshVar())) item |> Result.map (fun value -> name, value)) |> collectResults |> Result.map (Hir.Record >> make) | EFieldGet(target, name, _) -> - lowerExpression state environment (expressionType state target (TRecord Map.empty)) target |> Result.map (fun value -> make (Hir.FieldGet(value, name))) + match target with + | EVar(moduleName, _) -> + match environment |> Map.tryFind $"{moduleName}.{name}" with + | Some id -> Ok(make (Hir.Reference id)) + | None -> lowerExpression state environment (expressionType state target (TRecord Map.empty)) target |> Result.map (fun value -> make (Hir.FieldGet(value, name))) + | _ -> lowerExpression state environment (expressionType state target (TRecord Map.empty)) target |> Result.map (fun value -> make (Hir.FieldGet(value, name))) | ESome(item, _) -> let itemType = match inferred with TOption value -> value | _ -> Types.freshVar() lowerExpression state environment itemType item |> Result.map (Hir.SomeValue >> make) @@ -291,7 +296,12 @@ module HirLowering = constructorSymbols |> List.fold (fun env (typeName, caseName, id) -> env |> Map.add caseName id |> Map.add $"{typeName}.{caseName}" id) topLevel let mutable declarations = - constructorSymbols |> List.map (fun (typeName, caseName, id) -> + let builtinDeclarations = + BuiltinSignatures.builtinSchemes |> Map.toList |> List.sortBy fst |> List.map (fun (name, scheme) -> + let (Forall(_, valueType)) = scheme + let span = Span.mk (Span.pos 0 0) (Span.pos 0 0) + ({ Id = builtins.[name]; Name = name; Scheme = scheme; IsRecursive = false; IsExported = false; Span = span; Value = { Type = valueType; Span = span; Node = Hir.Intrinsic name } }: Hir.Declaration)) + builtinDeclarations @ (constructorSymbols |> List.map (fun (typeName, caseName, id) -> let definition, unionType = unionDefinitions |> List.find (fun (definition, _) -> definition.Name = typeName) let payloadType = definition.Cases |> List.find (fst >> (=) caseName) |> snd |> Option.map typeFromRef let value = @@ -301,7 +311,7 @@ module HirLowering = let parameter = newSymbol state let body = ({ Type = unionType; Span = definition.Span; Node = Hir.UnionCase(typeName, caseName, Some({ Type = itemType; Span = definition.Span; Node = Hir.Reference parameter })) }: Hir.Expression) ({ Type = TFun(itemType, unionType); Span = definition.Span; Node = Hir.Lambda(parameter, "value", itemType, body) }: Hir.Expression) - ({ Id = id; Name = $"{typeName}.{caseName}"; Scheme = Types.generalize Map.empty value.Type; IsRecursive = false; IsExported = false; Span = definition.Span; Value = value }: Hir.Declaration)) + ({ Id = id; Name = $"{typeName}.{caseName}"; Scheme = Types.generalize Map.empty value.Type; IsRecursive = false; IsExported = false; Span = definition.Span; Value = value }: Hir.Declaration))) let mutable entry = None let mutable errors = [] for statement in resolved.TypedProgram do diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index e2aee56..63e20cb 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -48,6 +48,7 @@ module HirValidation = | Hir.UnionCase(_, _, payload) -> payload |> Option.map references |> Option.defaultValue Set.empty | Hir.Map entries -> entries |> List.map (function Hir.KeyValue(key, value) -> Set.union (references key) (references value) | Hir.Spread value -> references value) |> Set.unionMany + | Hir.Intrinsic _ -> Set.empty let declared = declarationIds |> Set.ofList let unresolved = [ yield! program.Declarations |> List.collect (fun value -> references value.Value |> Set.toList) diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index 68404c3..f15c093 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -184,6 +184,7 @@ module IrArtifact = | Lir.Spread value -> writer.WriteString("kind", "spread"); writer.WritePropertyName("value"); writeExpression writer value writer.WriteEndObject() writer.WriteEndArray() + | Lir.Intrinsic name -> writer.WriteString("kind", "intrinsic"); writer.WriteString("name", name) writer.WriteEndObject() let serialize (program: Lir.Program) = @@ -315,6 +316,7 @@ module IrArtifact = | "key-value" -> Lir.KeyValue(readExpression (property "key" item), readExpression (property "value" item)) | "spread" -> Lir.Spread(readExpression (property "value" item)) | kind -> invalidOp $"Unknown map entry '{kind}'.") |> Seq.toList) + | "intrinsic" -> Lir.Intrinsic(stringProperty "name" element) | kind -> invalidOp $"Unknown LIR expression kind '{kind}'." { Type = expressionType; Span = span; Node = node } diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index c6a06ac..7e85bdc 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -47,6 +47,7 @@ module Lir = | Interpolate of InterpolationPart list | UnionCase of string * string * Expression option | Map of MapEntry list + | Intrinsic of string and MapEntry = | KeyValue of Expression * Expression @@ -158,6 +159,7 @@ module LirLowering = | Hir.Interpolate parts -> Lir.Interpolate(parts |> List.map (function Hir.TextPart text -> Lir.TextPart text | Hir.ValuePart item -> Lir.ValuePart(expression item))) | Hir.UnionCase(typeName, caseName, payload) -> Lir.UnionCase(typeName, caseName, Option.map expression payload) | Hir.Map entries -> Lir.Map(entries |> List.map (function Hir.KeyValue(key, item) -> Lir.KeyValue(expression key, expression item) | Hir.Spread item -> Lir.Spread(expression item))) + | Hir.Intrinsic name -> Lir.Intrinsic name { Type = value.Type; Span = value.Span; Node = node } let lower (program: Hir.Program) : Lir.Program = diff --git a/src/FScript.Compiler/LirInterpreter.fs b/src/FScript.Compiler/LirInterpreter.fs index be28c96..7034eff 100644 --- a/src/FScript.Compiler/LirInterpreter.fs +++ b/src/FScript.Compiler/LirInterpreter.fs @@ -18,6 +18,7 @@ type LirValue = | VOption of LirValue option | VUnion of string * string * LirValue option | VMap of (LirValue * LirValue) list + | VIntrinsic of string * LirValue list | VClosure of Lir.SymbolId * string * Lir.Expression * Map ref type ExecutionResult = { Value: LirValue } @@ -55,10 +56,49 @@ module LirInterpreter = | VMap entries -> entries |> List.map (fun (key, value) -> $"[{formatValue key}] = {formatValue value}") |> String.concat "; " |> fun value -> "{ " + value + " }" | VClosure _ -> "" + | VIntrinsic _ -> "" let private fail code span message = Error [ { Code = code; Stage = Interpreter; Message = message; Span = Some span } ] + let private intrinsicArity name = + let (Forall(_, valueType)) = FScript.Language.BuiltinSignatures.builtinSchemes.[name] + let rec count = function TFun(_, result) -> 1 + count result | _ -> 0 + count valueType + + let private invokeIntrinsic span name arguments = + let unsupported () = fail "FSI3020" span $"Intrinsic '{name}' is not yet executable in LIR." + match name, arguments with + | "ignore", [ _ ] -> Ok VUnit + | "Int.tryParse", [ VString value ] -> match System.Int64.TryParse value with true, parsed -> Ok(VOption(Some(VInt parsed))) | _ -> Ok(VOption None) + | "Float.tryParse", [ VString value ] -> match System.Double.TryParse(value, System.Globalization.NumberStyles.Float, System.Globalization.CultureInfo.InvariantCulture) with true, parsed -> Ok(VOption(Some(VFloat parsed))) | _ -> Ok(VOption None) + | "Bool.tryParse", [ VString value ] -> match System.Boolean.TryParse value with true, parsed -> Ok(VOption(Some(VBool parsed))) | _ -> Ok(VOption None) + | "Int.toString", [ VInt value ] -> Ok(VString(string value)) + | "Float.toString", [ VFloat value ] -> Ok(VString(value.ToString("R", System.Globalization.CultureInfo.InvariantCulture))) + | "Bool.toString", [ VBool value ] -> Ok(VString(value.ToString().ToLowerInvariant())) + | "String.replace", [ VString oldValue; VString newValue; VString value ] -> Ok(VString(value.Replace(oldValue, newValue))) + | "String.indexOf", [ VString needle; VString value ] -> let index = value.IndexOf(needle, System.StringComparison.Ordinal) in Ok(VOption(if index < 0 then None else Some(VInt(int64 index)))) + | "String.toLower", [ VString value ] -> Ok(VString(value.ToLowerInvariant())) + | "String.toUpper", [ VString value ] -> Ok(VString(value.ToUpperInvariant())) + | "String.substring", [ VInt start; VInt length; VString value ] when start >= 0L && length >= 0L && start + length <= int64 value.Length -> Ok(VOption(Some(VString(value.Substring(int start, int length))))) + | "String.substring", [ VInt _; VInt _; VString _ ] -> Ok(VOption None) + | "String.concat", [ VString separator; VList values ] -> values |> List.choose (function VString value -> Some value | _ -> None) |> String.concat separator |> VString |> Ok + | "String.split", [ VString separator; VString value ] -> value.Split([| separator |], System.StringSplitOptions.None) |> Array.toList |> List.map VString |> VList |> Ok + | "String.endsWith", [ VString suffix; VString value ] -> Ok(VBool(value.EndsWith(suffix, System.StringComparison.Ordinal))) + | "List.length", [ VList values ] -> Ok(VInt(int64 values.Length)) + | "List.rev", [ VList values ] -> Ok(VList(List.rev values)) + | "List.tryHead", [ VList values ] -> Ok(VOption(List.tryHead values)) + | "List.tail", [ VList(_ :: tail) ] -> Ok(VList tail) + | "List.append", [ VList left; VList right ] -> Ok(VList(left @ right)) + | "List.contains", [ value; VList values ] -> Ok(VBool(List.exists (equal value) values)) + | "Option.isNone", [ VOption value ] -> Ok(VBool value.IsNone) + | "Option.isSome", [ VOption value ] -> Ok(VBool value.IsSome) + | "Option.defaultValue", [ fallback; VOption value ] -> Ok(Option.defaultValue fallback value) + | "Map.count", [ VMap entries ] -> Ok(VInt(int64 entries.Length)) + | "Map.containsKey", [ key; VMap entries ] -> Ok(VBool(entries |> List.exists (fst >> equal key))) + | "Map.tryGet", [ key; VMap entries ] -> Ok(VOption(entries |> List.tryPick (fun (itemKey, value) -> if equal itemKey key then Some value else None))) + | _ -> unsupported () + let rec private matchPattern (pattern: Lir.Pattern) value = let combine left right = match left, right with @@ -101,6 +141,10 @@ module LirInterpreter = | Lir.CallClosure(fn, argument) -> match evaluate environment fn, evaluate environment argument with | Ok(VClosure(parameter, _, body, captured)), Ok value -> evaluate (captured.Value |> Map.add parameter value) body + | Ok(VIntrinsic(name, existing)), Ok value -> + let arguments = existing @ [ value ] + if arguments.Length = intrinsicArity name then invokeIntrinsic expression.Span name arguments + else Ok(VIntrinsic(name, arguments)) | Ok _, Ok _ -> fail "FSI3003" expression.Span "Attempted to call a non-function value." | Error a, Error b -> Error(a @ b) | Error errors, _ | _, Error errors -> Error errors @@ -234,6 +278,7 @@ module LirInterpreter = | Ok _ -> fail "FSI3017" expression.Span "Map spread requires a map." | Error errors -> Error errors build [] entries + | Lir.Intrinsic name -> Ok(VIntrinsic(name, [])) and private evaluateMany environment items = items |> List.map (evaluate environment) |> collect diff --git a/tests/FScript.Compiler.Tests/GoBackendTests.fs b/tests/FScript.Compiler.Tests/GoBackendTests.fs index 24ce3e2..09d6cfd 100644 --- a/tests/FScript.Compiler.Tests/GoBackendTests.fs +++ b/tests/FScript.Compiler.Tests/GoBackendTests.fs @@ -22,6 +22,9 @@ type GoBackendTests() = [ 0\n | Circle r -> r\nradius (Circle 3)", "3")>] [] [] + [] + [] + [] member _.``Generated projects run with Go`` (source: string, expected: string) = let input = { Program = Parser.parseProgram source; Externs = [] } let compiled = diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index aa10544..1827a41 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -120,3 +120,15 @@ type HirTests() = match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VBool true)) | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + [] + [] + [] + member _.``Executes portable scalar and collection intrinsics`` (source: string, expected: string) = + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VString expected)) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) From 7889dc2e18c11442ae4c80953d4ecf9ed438fd06 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 17:13:55 +0200 Subject: [PATCH 016/133] feat(compiler): retain inert host extern linkage --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 ++ src/FScript.Compiler/Compiler.fs | 2 +- src/FScript.Compiler/GoBackend.fs | 13 +++++++++++++ src/FScript.Compiler/Hir.fs | 2 ++ src/FScript.Compiler/HirLowering.fs | 12 +++++++++--- src/FScript.Compiler/HirValidation.fs | 1 + src/FScript.Compiler/IrArtifact.fs | 2 ++ src/FScript.Compiler/Lir.fs | 2 ++ src/FScript.Compiler/LirInterpreter.fs | 7 +++++++ tests/FScript.Compiler.Tests/HirTests.fs | 21 +++++++++++++++++++++ 11 files changed, 61 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b0baf72..c6ebaea 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added serializable host-extern linkage that remains inert when unused and reports a dedicated LIR diagnostic when reached. - Added explicit portable intrinsic declarations and LIR/Go execution for scalar parsing and formatting plus core string, list, option, and map queries. - Added atomic top-level mutual-recursive closure initialization in the LIR interpreter and generated Go projects. - Added deterministic ordered-map construction, left-precedence spreading, indexing, equality, serialization, and Go runtime support. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 1561c26..ca8e315 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -28,6 +28,8 @@ Top-level mutual-recursive declarations are initialized as an atomic closure gro Language builtins are explicit `Intrinsic` declarations rather than magic unresolved references. Consumers accumulate their curried arguments according to the catalog signature and dispatch only at full application. The initial portable implementations cover scalar parsing/formatting, strings, simple list and option operations, and map queries; a reachable catalog entry without an implementation receives a stage-specific unsupported diagnostic or runtime failure. +Host externs are copied into IR as name, signature, and arity only; their executable .NET delegates never cross the compiler boundary. Constructing or retaining an unused extern handle is inert. Fully applying one in the LIR interpreter produces `FSI3021`, preserving the rule that unreachable unsupported linkage does not block execution. + Serialized `.fsir` files contain one closed resolved program plus explicit foreign linkage declarations. JSON format versions are internal and may be rejected rather than upgraded. ## Representation policy diff --git a/src/FScript.Compiler/Compiler.fs b/src/FScript.Compiler/Compiler.fs index 55e1497..1c474c5 100644 --- a/src/FScript.Compiler/Compiler.fs +++ b/src/FScript.Compiler/Compiler.fs @@ -15,7 +15,7 @@ module Compiler = try input.Program |> TypeInfer.inferResolvedProgramWithExterns input.Externs - |> HirLowering.lower + |> HirLowering.lower input.Externs |> Result.bind HirValidation.validate |> Result.bind (fun hir -> hir |> LirLowering.lower |> LirValidation.validate diff --git a/src/FScript.Compiler/GoBackend.fs b/src/FScript.Compiler/GoBackend.fs index c06c789..68a9319 100644 --- a/src/FScript.Compiler/GoBackend.fs +++ b/src/FScript.Compiler/GoBackend.fs @@ -123,6 +123,18 @@ module GoBackend = cast resultType $"fsruntime.Intrinsic({quoted}, []any{{{values}}})" build 0 [] intrinsicType + let private emitHostExternal name externalType = + let quoted = System.Text.Json.JsonSerializer.Serialize(name) + let rec build index arguments = function + | TFun(argumentType, resultType) -> + let argument = $"externalArg{index}" + let body = build (index + 1) (arguments @ [ argument ]) resultType + $"func({argument} {goType argumentType}) {goType resultType} {{ return {body} }}" + | resultType -> + let values = String.concat ", " arguments + cast resultType $"fsruntime.Intrinsic(\"__host:\" + {quoted}, []any{{{values}}})" + build 0 [] externalType + let rec private emitExpression environment (expression: Lir.Expression) = match expression.Node with | Lir.Constant value -> Ok(constant value) @@ -247,6 +259,7 @@ module GoBackend = let joined = String.concat ", " parts $"fsruntime.BuildMap([]fsruntime.MapPart{{{joined}}})") | Lir.Intrinsic name -> Ok(emitIntrinsic name expression.Type) + | Lir.HostExternal(name, _) -> Ok(emitHostExternal name expression.Type) | Lir.Operation _ -> Error "Invalid operation arity" and private collect<'a> (results: Result<'a, string> list) : Result<'a list, string> = diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs index c42215a..cc74327 100644 --- a/src/FScript.Compiler/Hir.fs +++ b/src/FScript.Compiler/Hir.fs @@ -60,6 +60,7 @@ module Hir = | UnionCase of string * string * Expression option | Map of MapEntry list | Intrinsic of string + | HostExternal of string * int and MapEntry = | KeyValue of Expression * Expression @@ -177,6 +178,7 @@ module Hir = lines.Add(typed "map") entries |> List.iter (function KeyValue(key, value) -> expression (indent + 2) key; expression (indent + 2) value | Spread value -> expression (indent + 2) value) | Intrinsic name -> lines.Add(typed $"intrinsic {name}") + | HostExternal(name, arity) -> lines.Add(typed $"host-extern {name}/{arity}") for declaration in program.Declarations do lines.Add(sprintf "decl %%%d %s: %s" (symbolNumber declaration.Id) declaration.Name (formatType (let (Forall(_, value)) = declaration.Scheme in value))) diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index 32c8cf0..98e32c4 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -251,7 +251,7 @@ module HirLowering = |> collectResults |> Result.map (Hir.Map >> make) | _ -> Error [ Diagnostic.hir "FSH1099" $"HIR lowering does not yet support {expression.GetType().Name}." (Some span) ] - let lower (resolved: TypeInfer.ResolvedProgram) : Result = + let lower (externs: ExternalFunction list) (resolved: TypeInfer.ResolvedProgram) : Result = let expressionTypes = resolved.ExpressionTypes |> List.fold (fun state item -> state |> Map.add item.Expr item.Type) Map.empty @@ -262,6 +262,7 @@ module HirLowering = |> List.sortBy fst |> List.map (fun (name, _) -> name, newSymbol state) |> Map.ofList + let externSymbols = externs |> List.sortBy _.Name |> List.map (fun value -> value.Name, newSymbol state) |> Map.ofList let typeDefinitions = resolved.TypedProgram |> List.choose (function TypeInfer.TSType definition -> Some definition | _ -> None) let rec typeFromRef = function @@ -291,7 +292,7 @@ module HirLowering = | TypeInfer.TSLetRecGroup(bindings, _, _) -> bindings |> List.map (fun (name, _, _, _) -> name) | _ -> []) |> List.distinct - |> List.fold (fun env name -> env |> Map.add name (newSymbol state)) builtins + |> List.fold (fun env name -> env |> Map.add name (newSymbol state)) (Map.fold (fun env name id -> Map.add name id env) builtins externSymbols) let topLevel = constructorSymbols |> List.fold (fun env (typeName, caseName, id) -> env |> Map.add caseName id |> Map.add $"{typeName}.{caseName}" id) topLevel @@ -301,7 +302,12 @@ module HirLowering = let (Forall(_, valueType)) = scheme let span = Span.mk (Span.pos 0 0) (Span.pos 0 0) ({ Id = builtins.[name]; Name = name; Scheme = scheme; IsRecursive = false; IsExported = false; Span = span; Value = { Type = valueType; Span = span; Node = Hir.Intrinsic name } }: Hir.Declaration)) - builtinDeclarations @ (constructorSymbols |> List.map (fun (typeName, caseName, id) -> + let externalDeclarations = + externs |> List.sortBy _.Name |> List.map (fun external -> + let (Forall(_, valueType)) = external.Scheme + let span = Span.mk (Span.pos 0 0) (Span.pos 0 0) + ({ Id = externSymbols.[external.Name]; Name = external.Name; Scheme = external.Scheme; IsRecursive = false; IsExported = false; Span = span; Value = { Type = valueType; Span = span; Node = Hir.HostExternal(external.Name, external.Arity) } }: Hir.Declaration)) + builtinDeclarations @ externalDeclarations @ (constructorSymbols |> List.map (fun (typeName, caseName, id) -> let definition, unionType = unionDefinitions |> List.find (fun (definition, _) -> definition.Name = typeName) let payloadType = definition.Cases |> List.find (fst >> (=) caseName) |> snd |> Option.map typeFromRef let value = diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index 63e20cb..7c06891 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -49,6 +49,7 @@ module HirValidation = | Hir.Map entries -> entries |> List.map (function Hir.KeyValue(key, value) -> Set.union (references key) (references value) | Hir.Spread value -> references value) |> Set.unionMany | Hir.Intrinsic _ -> Set.empty + | Hir.HostExternal _ -> Set.empty let declared = declarationIds |> Set.ofList let unresolved = [ yield! program.Declarations |> List.collect (fun value -> references value.Value |> Set.toList) diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index f15c093..d40e6c5 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -185,6 +185,7 @@ module IrArtifact = writer.WriteEndObject() writer.WriteEndArray() | Lir.Intrinsic name -> writer.WriteString("kind", "intrinsic"); writer.WriteString("name", name) + | Lir.HostExternal(name, arity) -> writer.WriteString("kind", "host-external"); writer.WriteString("name", name); writer.WriteNumber("arity", arity) writer.WriteEndObject() let serialize (program: Lir.Program) = @@ -317,6 +318,7 @@ module IrArtifact = | "spread" -> Lir.Spread(readExpression (property "value" item)) | kind -> invalidOp $"Unknown map entry '{kind}'.") |> Seq.toList) | "intrinsic" -> Lir.Intrinsic(stringProperty "name" element) + | "host-external" -> Lir.HostExternal(stringProperty "name" element, (property "arity" element).GetInt32()) | kind -> invalidOp $"Unknown LIR expression kind '{kind}'." { Type = expressionType; Span = span; Node = node } diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index 7e85bdc..31c886a 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -48,6 +48,7 @@ module Lir = | UnionCase of string * string * Expression option | Map of MapEntry list | Intrinsic of string + | HostExternal of string * int and MapEntry = | KeyValue of Expression * Expression @@ -160,6 +161,7 @@ module LirLowering = | Hir.UnionCase(typeName, caseName, payload) -> Lir.UnionCase(typeName, caseName, Option.map expression payload) | Hir.Map entries -> Lir.Map(entries |> List.map (function Hir.KeyValue(key, item) -> Lir.KeyValue(expression key, expression item) | Hir.Spread item -> Lir.Spread(expression item))) | Hir.Intrinsic name -> Lir.Intrinsic name + | Hir.HostExternal(name, arity) -> Lir.HostExternal(name, arity) { Type = value.Type; Span = value.Span; Node = node } let lower (program: Hir.Program) : Lir.Program = diff --git a/src/FScript.Compiler/LirInterpreter.fs b/src/FScript.Compiler/LirInterpreter.fs index 7034eff..a485d03 100644 --- a/src/FScript.Compiler/LirInterpreter.fs +++ b/src/FScript.Compiler/LirInterpreter.fs @@ -19,6 +19,7 @@ type LirValue = | VUnion of string * string * LirValue option | VMap of (LirValue * LirValue) list | VIntrinsic of string * LirValue list + | VExternal of string * int * LirValue list | VClosure of Lir.SymbolId * string * Lir.Expression * Map ref type ExecutionResult = { Value: LirValue } @@ -57,6 +58,7 @@ module LirInterpreter = entries |> List.map (fun (key, value) -> $"[{formatValue key}] = {formatValue value}") |> String.concat "; " |> fun value -> "{ " + value + " }" | VClosure _ -> "" | VIntrinsic _ -> "" + | VExternal _ -> "" let private fail code span message = Error [ { Code = code; Stage = Interpreter; Message = message; Span = Some span } ] @@ -145,6 +147,10 @@ module LirInterpreter = let arguments = existing @ [ value ] if arguments.Length = intrinsicArity name then invokeIntrinsic expression.Span name arguments else Ok(VIntrinsic(name, arguments)) + | Ok(VExternal(name, arity, existing)), Ok value -> + let arguments = existing @ [ value ] + if arguments.Length = arity then fail "FSI3021" expression.Span $"Host external '{name}' is not executable from LIR." + else Ok(VExternal(name, arity, arguments)) | Ok _, Ok _ -> fail "FSI3003" expression.Span "Attempted to call a non-function value." | Error a, Error b -> Error(a @ b) | Error errors, _ | _, Error errors -> Error errors @@ -279,6 +285,7 @@ module LirInterpreter = | Error errors -> Error errors build [] entries | Lir.Intrinsic name -> Ok(VIntrinsic(name, [])) + | Lir.HostExternal(name, arity) -> Ok(VExternal(name, arity, [])) and private evaluateMany environment items = items |> List.map (evaluate environment) |> collect diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index 1827a41..be56692 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -132,3 +132,24 @@ type HirTests() = match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VString expected)) | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``Retains host linkage but rejects only reachable external calls`` () = + let external : ExternalFunction = + { Name = "Host.echo" + Scheme = Forall([], TFun(TString, TString)) + Arity = 1 + Impl = fun _ values -> values.Head } + let compile source = Compiler.compile { Program = Parser.parseProgram source; Externs = [ external ] } + match compile "1" with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 1L)) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + match compile "Host.echo \"value\"" with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok _ -> Assert.Fail("Expected unsupported external diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSI3021")) From 2634b341995876fd8f058f7ebbf3ad2ba4ea02bb Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 17:16:21 +0200 Subject: [PATCH 017/133] refactor(compiler): split LIR consumers into assemblies --- CHANGELOG.md | 1 + FScript.sln | 30 +++++++++++++++++++ docs/architecture/assemblies-and-roles.md | 29 ++++++++++++++++-- docs/architecture/typed-ir-and-backends.md | 4 ++- .../FScript.Backend.Go.fsproj | 12 ++++++++ .../GoBackend.fs | 0 src/FScript.Compiler/FScript.Compiler.fsproj | 2 -- .../FScript.Interpreter.Lir.fsproj | 12 ++++++++ .../LirInterpreter.fs | 0 src/FScript/FScript.fsproj | 2 ++ .../FScript.Compiler.Tests.fsproj | 2 ++ 11 files changed, 88 insertions(+), 6 deletions(-) create mode 100644 src/FScript.Backend.Go/FScript.Backend.Go.fsproj rename src/{FScript.Compiler => FScript.Backend.Go}/GoBackend.fs (100%) create mode 100644 src/FScript.Interpreter.Lir/FScript.Interpreter.Lir.fsproj rename src/{FScript.Compiler => FScript.Interpreter.Lir}/LirInterpreter.fs (100%) diff --git a/CHANGELOG.md b/CHANGELOG.md index c6ebaea..8e005ec 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Split LIR execution and Go generation into independent `FScript.Interpreter.Lir` and `FScript.Backend.Go` assemblies. - Added serializable host-extern linkage that remains inert when unused and reports a dedicated LIR diagnostic when reached. - Added explicit portable intrinsic declarations and LIR/Go execution for scalar parsing and formatting plus core string, list, option, and map queries. - Added atomic top-level mutual-recursive closure initialization in the LIR interpreter and generated Go projects. diff --git a/FScript.sln b/FScript.sln index ba48c36..6c66187 100644 --- a/FScript.sln +++ b/FScript.sln @@ -35,6 +35,10 @@ Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "FScript.TypeProvider.Tests" EndProject Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "FScript.JavaScript", "src\FScript.JavaScript\FScript.JavaScript.fsproj", "{CEB16446-D96F-40A7-9771-F90B9A78D7B5}" EndProject +Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "FScript.Interpreter.Lir", "src\FScript.Interpreter.Lir\FScript.Interpreter.Lir.fsproj", "{40BB6F66-433C-407A-B71B-AF8A7DD2992A}" +EndProject +Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "FScript.Backend.Go", "src\FScript.Backend.Go\FScript.Backend.Go.fsproj", "{FC0FF53F-A10D-40B2-9C30-87AF93AD8A52}" +EndProject Global GlobalSection(SolutionConfigurationPlatforms) = preSolution Debug|Any CPU = Debug|Any CPU @@ -213,6 +217,30 @@ Global {CEB16446-D96F-40A7-9771-F90B9A78D7B5}.Release|x64.Build.0 = Release|Any CPU {CEB16446-D96F-40A7-9771-F90B9A78D7B5}.Release|x86.ActiveCfg = Release|Any CPU {CEB16446-D96F-40A7-9771-F90B9A78D7B5}.Release|x86.Build.0 = Release|Any CPU + {40BB6F66-433C-407A-B71B-AF8A7DD2992A}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {40BB6F66-433C-407A-B71B-AF8A7DD2992A}.Debug|Any CPU.Build.0 = Debug|Any CPU + {40BB6F66-433C-407A-B71B-AF8A7DD2992A}.Debug|x64.ActiveCfg = Debug|Any CPU + {40BB6F66-433C-407A-B71B-AF8A7DD2992A}.Debug|x64.Build.0 = Debug|Any CPU + {40BB6F66-433C-407A-B71B-AF8A7DD2992A}.Debug|x86.ActiveCfg = Debug|Any CPU + {40BB6F66-433C-407A-B71B-AF8A7DD2992A}.Debug|x86.Build.0 = Debug|Any CPU + {40BB6F66-433C-407A-B71B-AF8A7DD2992A}.Release|Any CPU.ActiveCfg = Release|Any CPU + {40BB6F66-433C-407A-B71B-AF8A7DD2992A}.Release|Any CPU.Build.0 = Release|Any CPU + {40BB6F66-433C-407A-B71B-AF8A7DD2992A}.Release|x64.ActiveCfg = Release|Any CPU + {40BB6F66-433C-407A-B71B-AF8A7DD2992A}.Release|x64.Build.0 = Release|Any CPU + {40BB6F66-433C-407A-B71B-AF8A7DD2992A}.Release|x86.ActiveCfg = Release|Any CPU + {40BB6F66-433C-407A-B71B-AF8A7DD2992A}.Release|x86.Build.0 = Release|Any CPU + {FC0FF53F-A10D-40B2-9C30-87AF93AD8A52}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {FC0FF53F-A10D-40B2-9C30-87AF93AD8A52}.Debug|Any CPU.Build.0 = Debug|Any CPU + {FC0FF53F-A10D-40B2-9C30-87AF93AD8A52}.Debug|x64.ActiveCfg = Debug|Any CPU + {FC0FF53F-A10D-40B2-9C30-87AF93AD8A52}.Debug|x64.Build.0 = Debug|Any CPU + {FC0FF53F-A10D-40B2-9C30-87AF93AD8A52}.Debug|x86.ActiveCfg = Debug|Any CPU + {FC0FF53F-A10D-40B2-9C30-87AF93AD8A52}.Debug|x86.Build.0 = Debug|Any CPU + {FC0FF53F-A10D-40B2-9C30-87AF93AD8A52}.Release|Any CPU.ActiveCfg = Release|Any CPU + {FC0FF53F-A10D-40B2-9C30-87AF93AD8A52}.Release|Any CPU.Build.0 = Release|Any CPU + {FC0FF53F-A10D-40B2-9C30-87AF93AD8A52}.Release|x64.ActiveCfg = Release|Any CPU + {FC0FF53F-A10D-40B2-9C30-87AF93AD8A52}.Release|x64.Build.0 = Release|Any CPU + {FC0FF53F-A10D-40B2-9C30-87AF93AD8A52}.Release|x86.ActiveCfg = Release|Any CPU + {FC0FF53F-A10D-40B2-9C30-87AF93AD8A52}.Release|x86.Build.0 = Release|Any CPU EndGlobalSection GlobalSection(SolutionProperties) = preSolution HideSolutionNode = FALSE @@ -232,5 +260,7 @@ Global {14D91D30-8E5E-482A-940B-CC55F2DE80AA} = {827E0CD3-B72D-47B6-A68D-7590B98EB39B} {42D043DE-8987-4072-8841-DCB2144AC18C} = {0AB3BF05-4346-4AA6-1389-037BE0695223} {CEB16446-D96F-40A7-9771-F90B9A78D7B5} = {827E0CD3-B72D-47B6-A68D-7590B98EB39B} + {40BB6F66-433C-407A-B71B-AF8A7DD2992A} = {827E0CD3-B72D-47B6-A68D-7590B98EB39B} + {FC0FF53F-A10D-40B2-9C30-87AF93AD8A52} = {827E0CD3-B72D-47B6-A68D-7590B98EB39B} EndGlobalSection EndGlobal diff --git a/docs/architecture/assemblies-and-roles.md b/docs/architecture/assemblies-and-roles.md index 4b68306..8956b88 100644 --- a/docs/architecture/assemblies-and-roles.md +++ b/docs/architecture/assemblies-and-roles.md @@ -40,16 +40,37 @@ Use this when: ### `FScript.Compiler` Role: -- Reusable .NET compiler pipeline for typed HIR, executable LIR, IR artifacts, interpretation, and source backends. +- Reusable .NET compiler pipeline for typed HIR, executable LIR, and IR artifacts. Responsibilities: - Consume the resolved typed result produced by `FScript.Language`. - Validate and lower HIR to backend-independent LIR. - Serialize and validate whole-program `.fsir` artifacts. -- Execute supported LIR and generate deterministic Go source projects. `FScript.Compiler` depends on `FScript.Language` and is intentionally excluded from the Fable JavaScript package. +### `FScript.Interpreter.Lir` +Role: +- Backend-independent execution engine for validated LIR. + +Responsibilities: +- Execute LIR values, closures, decision trees, aggregates, intrinsics, and failures. +- Supply script identity and arguments through `InterpreterContext`. +- Reject reachable unsupported target or host linkage with interpreter diagnostics. + +`FScript.Interpreter.Lir` depends on `FScript.Compiler`; the core compiler has no dependency on the interpreter. + +### `FScript.Backend.Go` +Role: +- Go 1.25 source backend for validated LIR. + +Responsibilities: +- Select Go representations for LIR values. +- Generate deterministic Go project files and the embedded FScript Go runtime. +- Report target-specific `FSB4xxx` diagnostics without adding Go concepts to HIR or LIR. + +`FScript.Backend.Go` depends on `FScript.Compiler`; the core compiler has no dependency on the Go backend or a Go installation. + NuGet: - `MagnusOpera.FScript.Language` @@ -165,11 +186,13 @@ Use this when: ## Dependency direction - `FScript.Language` has no dependency on `FScript.Runtime`. +- `FScript.Compiler` depends only on `FScript.Language`. +- `FScript.Interpreter.Lir` and `FScript.Backend.Go` each depend on `FScript.Compiler`. - `FScript.JavaScript` depends on `FScript.Language` and Fable packages. - `FScript.Runtime` depends on `FScript.Language` types. - `FScript.CSharpInterop` depends on both `FScript.Language` and `FScript.Runtime`. - `FScript.LanguageServer` depends on `FScript.CSharpInterop`. - `FScript.TypeProvider` depends on `FScript.Language` and `FScript.Runtime`. -- `FScript` depends on both `FScript.Language` and `FScript.Runtime`. +- `FScript` depends on `FScript.Language`, `FScript.Runtime`, `FScript.Compiler`, `FScript.Interpreter.Lir`, and `FScript.Backend.Go`. This keeps the language engine reusable while runtime capabilities remain host-configurable. The JavaScript facade is intentionally separate from `.NET` runtime externs: v1 exposes core language hosting, virtual imports, tagged values, and JS-provided externs, while filesystem, console, task, and other runtime modules stay in the .NET runtime layer. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index ca8e315..a78ced5 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -12,7 +12,9 @@ source -> parser/includes -> inference + resolved typed tree -> HIR -> LIR `FScript.Language` remains the parser, type-system, legacy evaluator, and Fable-compatible package. It emits a resolved typed tree in the same inference pass as its compatibility `TypedProgram`. -`FScript.Compiler` is a reusable, initially non-packaged .NET assembly. It owns HIR, HIR validation, lowering, LIR, LIR validation, artifact serialization, LIR interpretation, and target backends. It depends on `FScript.Language`; the language assembly never depends on the compiler. +`FScript.Compiler` is a reusable, initially non-packaged .NET assembly. It owns HIR, HIR validation, lowering, LIR, LIR validation, and artifact serialization. It depends on `FScript.Language`; the language assembly never depends on the compiler. + +Validated LIR consumers are separate leaf assemblies. `FScript.Interpreter.Lir` owns execution and runtime LIR values. `FScript.Backend.Go` owns representation selection, Go emission, and embedded runtime templates. Both reference `FScript.Compiler`, while the compiler references neither consumer. Public modules retain the `FScript.Compiler` namespace so the API surface remains source-compatible while assembly ownership is explicit. ## IR boundaries diff --git a/src/FScript.Backend.Go/FScript.Backend.Go.fsproj b/src/FScript.Backend.Go/FScript.Backend.Go.fsproj new file mode 100644 index 0000000..0f20dbc --- /dev/null +++ b/src/FScript.Backend.Go/FScript.Backend.Go.fsproj @@ -0,0 +1,12 @@ + + + net10.0 + false + + + + + + + + diff --git a/src/FScript.Compiler/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs similarity index 100% rename from src/FScript.Compiler/GoBackend.fs rename to src/FScript.Backend.Go/GoBackend.fs diff --git a/src/FScript.Compiler/FScript.Compiler.fsproj b/src/FScript.Compiler/FScript.Compiler.fsproj index 2c3c34e..e8f9104 100644 --- a/src/FScript.Compiler/FScript.Compiler.fsproj +++ b/src/FScript.Compiler/FScript.Compiler.fsproj @@ -9,9 +9,7 @@ - - diff --git a/src/FScript.Interpreter.Lir/FScript.Interpreter.Lir.fsproj b/src/FScript.Interpreter.Lir/FScript.Interpreter.Lir.fsproj new file mode 100644 index 0000000..3533c5b --- /dev/null +++ b/src/FScript.Interpreter.Lir/FScript.Interpreter.Lir.fsproj @@ -0,0 +1,12 @@ + + + net10.0 + false + + + + + + + + diff --git a/src/FScript.Compiler/LirInterpreter.fs b/src/FScript.Interpreter.Lir/LirInterpreter.fs similarity index 100% rename from src/FScript.Compiler/LirInterpreter.fs rename to src/FScript.Interpreter.Lir/LirInterpreter.fs diff --git a/src/FScript/FScript.fsproj b/src/FScript/FScript.fsproj index 2daeb1d..d859048 100644 --- a/src/FScript/FScript.fsproj +++ b/src/FScript/FScript.fsproj @@ -40,6 +40,8 @@ + + diff --git a/tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj b/tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj index f8317e3..83a18fb 100644 --- a/tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj +++ b/tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj @@ -17,5 +17,7 @@ + + From 365dfda10cb895c3cacd596bb28db3a930779849 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 17:22:48 +0200 Subject: [PATCH 018/133] feat(compiler): lower iteration and binding patterns --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 ++ docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Backend.Go/GoBackend.fs | 16 +++++++++- src/FScript.Compiler/Hir.fs | 5 ++++ src/FScript.Compiler/HirLowering.fs | 29 ++++++++++++++++++- src/FScript.Compiler/HirValidation.fs | 2 ++ src/FScript.Compiler/IrArtifact.fs | 8 +++++ src/FScript.Compiler/Lir.fs | 4 +++ src/FScript.Interpreter.Lir/LirInterpreter.fs | 14 +++++++++ .../FScript.Compiler.Tests/GoBackendTests.fs | 3 ++ tests/FScript.Compiler.Tests/HirTests.fs | 14 +++++++++ .../FScript.Compiler.Tests/IrArtifactTests.fs | 5 +++- 13 files changed, 101 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8e005ec..9ed365a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added compiled local binding patterns, sequential list iteration, and record/union type tokens across LIR artifacts, interpretation, and Go generation. - Split LIR execution and Go generation into independent `FScript.Interpreter.Lir` and `FScript.Backend.Go` assemblies. - Added serializable host-extern linkage that remains inert when unused and reports a dedicated LIR diagnostic when reached. - Added explicit portable intrinsic declarations and LIR/Go execution for scalar parsing and formatting plus core string, list, option, and map queries. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index a78ced5..7f9386b 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -32,6 +32,8 @@ Language builtins are explicit `Intrinsic` declarations rather than magic unreso Host externs are copied into IR as name, signature, and arity only; their executable .NET delegates never cross the compiler boundary. Constructing or retaining an unused extern handle is inert. Fully applying one in the LIR interpreter produces `FSI3021`, preserving the rule that unreachable unsupported linkage does not block execution. +Local binding patterns reuse the same single-scrutinee decision-tree representation as `match`. Sequential `for ... in ... do ...` is explicit in HIR and LIR so each consumer evaluates the input list once, visits values in order, and discards each body result before returning unit. `typeof` resolves declared record and union definitions during HIR construction and carries the semantic type as a serializable type-token node. + Serialized `.fsir` files contain one closed resolved program plus explicit foreign linkage declarations. JSON format versions are internal and may be rejected rather than upgraded. ## Representation policy diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index d21226b..5dab6fe 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -28,6 +28,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Declared records retain declaration identity for named annotations. Structural record annotations retain field-shape compatibility and may accept additional fields where current inference permits it. A backend must insert representation conversions rather than change compatibility rules. - Discriminated unions retain their declared type name, case name, and optional payload. Recursive types require `type rec`; mutually recursive type-definition graphs are rejected. - `typeof` produces an FScript type token and `nameof` produces the resolved source name. +- Compiled local pattern bindings evaluate their source once and use the ordinary match-failure path. Compiled `for` evaluates its list once, executes bodies from first to last, discards their values, and returns unit. Compiled `typeof` retains the resolved declared record or union shape in the artifact instead of performing a name lookup at execution time. ## Expressions, patterns, and equality diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index 68a9319..883aaeb 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -14,6 +14,7 @@ module GoBackend = | TFloat -> "float64" | TBool -> "bool" | TString -> "string" + | TTypeToken -> "string" | TVar _ -> "any" | TFun(argument, result) -> $"func({goType argument}) {goType result}" | TTuple _ -> "[]any" @@ -260,6 +261,15 @@ module GoBackend = $"fsruntime.BuildMap([]fsruntime.MapPart{{{joined}}})") | Lir.Intrinsic name -> Ok(emitIntrinsic name expression.Type) | Lir.HostExternal(name, _) -> Ok(emitHostExternal name expression.Type) + | Lir.ForEach(id, _, itemType, source, body) -> + let name = identifier "forValue" id + let scoped = environment |> Map.add id name + match emitExpression environment source, emitExpression scoped body with + | Ok sourceCode, Ok bodyCode -> + let itemCode = cast itemType "rawValue" + Ok $"func() struct{{}} {{ for _, rawValue := range fsruntime.List({sourceCode}) {{ {name} := {itemCode}; _ = {bodyCode} }}; return struct{{}}{{}} }}()" + | Error error, _ | _, Error error -> Error error + | Lir.TypeToken tokenType -> Ok(System.Text.Json.JsonSerializer.Serialize(Types.typeToString tokenType)) | Lir.Operation _ -> Error "Invalid operation arity" and private collect<'a> (results: Result<'a, string> list) : Result<'a list, string> = @@ -291,9 +301,13 @@ module GoBackend = | None, Ok entryCode -> let moduleName = if String.IsNullOrWhiteSpace options.ModuleName then "fscript.generated/program" else options.ModuleName let programSource = $"package fscriptprogram\n\nimport \"{moduleName}/internal/fsruntime\"\n\n{declarations}\nfunc init() {{\n{initializers}}}\n\nfunc Run() any {{ return {entryCode} }}\n" + let printExpression = + match program.Entry with + | Some { Type = TUnit } -> "_ = fsruntime.Format(fscriptprogram.Run()); fmt.Println(\"()\")" + | _ -> "fmt.Println(fsruntime.Format(fscriptprogram.Run()))" Ok [ { RelativePath = "go.mod"; Content = $"module {moduleName}\n\ngo 1.25\n" } - { RelativePath = "main.go"; Content = $"package main\n\nimport (\n\t\"fmt\"\n\t\"{moduleName}/internal/fscriptprogram\"\n\t\"{moduleName}/internal/fsruntime\"\n)\n\nfunc main() {{ fmt.Println(fsruntime.Format(fscriptprogram.Run())) }}\n" } + { RelativePath = "main.go"; Content = $"package main\n\nimport (\n\t\"fmt\"\n\t\"{moduleName}/internal/fscriptprogram\"\n\t\"{moduleName}/internal/fsruntime\"\n)\n\nfunc main() {{ {printExpression} }}\n" } { RelativePath = "internal/fscriptprogram/program.go"; Content = programSource } { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"sort\"\n\t\"strconv\"\n\t\"strings\"\n)\n\ntype Option struct { HasValue bool; Value any }\ntype Union struct { TypeName string; CaseName string; HasValue bool; Value any }\ntype MapEntry struct { Key any; Value any }\ntype OrderedMap struct { Entries []MapEntry }\ntype MapPart struct { IsSpread bool; Key any; Value any; Map OrderedMap }\n\nfunc Unit(value any) struct{} { return struct{}{} }\nfunc Int(value any) int64 { return value.(int64) }\nfunc Float(value any) float64 { return value.(float64) }\nfunc Bool(value any) bool { return value.(bool) }\nfunc String(value any) string { return value.(string) }\nfunc List(value any) []any { return value.([]any) }\nfunc Record(value any) map[string]any { return value.(map[string]any) }\nfunc AsOption(value any) Option { return value.(Option) }\nfunc AsUnion(value any) Union { return value.(Union) }\nfunc AsMap(value any) OrderedMap { return value.(OrderedMap) }\nfunc Some(value any) Option { return Option{HasValue: true, Value: value} }\nfunc None() Option { return Option{} }\nfunc IsSome(value any) bool { return AsOption(value).HasValue }\nfunc OptionValue(value any) any { return AsOption(value).Value }\nfunc MakeUnion(typeName string, caseName string) Union { return Union{TypeName: typeName, CaseName: caseName} }\nfunc MakeUnionValue(typeName string, caseName string, value any) Union { return Union{TypeName: typeName, CaseName: caseName, HasValue: true, Value: value} }\nfunc IsUnion(value any, typeName string, caseName string) bool { item := AsUnion(value); return item.TypeName == typeName && item.CaseName == caseName }\nfunc UnionHasValue(value any) bool { return AsUnion(value).HasValue }\nfunc UnionValue(value any) any { return AsUnion(value).Value }\nfunc HasField(value any, name string) bool { _, ok := Record(value)[name]; return ok }\nfunc Equal(left any, right any) bool { return reflect.DeepEqual(left, right) }\nfunc Cons(head any, tail []any) []any { return append([]any{head}, tail...) }\nfunc Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) }\nfunc Tail(value any) []any { return append([]any{}, List(value)[1:]...) }\nfunc Range(first int64, last int64) []any { step := int64(1); if first > last { step = -1 }; values := []any{}; for current := first; ; current += step { values = append(values, current); if current == last { break } }; return values }\nfunc UpdateRecord(value any, updates map[string]any) map[string]any { result := map[string]any{}; for key, item := range Record(value) { result[key] = item }; for key, item := range updates { result[key] = item }; return result }\nfunc Pair(key any, value any) MapPart { return MapPart{Key: key, Value: value} }\nfunc SpreadMap(value any) MapPart { return MapPart{IsSpread: true, Map: AsMap(value)} }\nfunc mapSet(entries []MapEntry, key any, value any, replace bool) []MapEntry { for index, entry := range entries { if Equal(entry.Key, key) { if replace { entries[index].Value = value }; return entries } }; return append(entries, MapEntry{Key: key, Value: value}) }\nfunc BuildMap(parts []MapPart) OrderedMap { entries := []MapEntry{}; for _, part := range parts { if part.IsSpread { for _, entry := range part.Map.Entries { entries = mapSet(entries, entry.Key, entry.Value, false) } } else { entries = mapSet(entries, part.Key, part.Value, true) } }; sort.SliceStable(entries, func(left int, right int) bool { return Format(entries[left].Key) < Format(entries[right].Key) }); return OrderedMap{Entries: entries} }\nfunc Index(value any, index any) Option { switch target := value.(type) { case []any: position := Int(index); if position < 0 || position >= int64(len(target)) { return None() }; return Some(target[position]); case OrderedMap: for _, entry := range target.Entries { if Equal(entry.Key, index) { return Some(entry.Value) } }; return None(); default: panic(\"Index access requires a list or map\") } }\nfunc Intrinsic(name string, args []any) any { switch name { case \"ignore\": return struct{}{}; case \"List.empty\": return []any{}; case \"Map.empty\": return OrderedMap{}; case \"Int.tryParse\": value, error := strconv.ParseInt(String(args[0]), 10, 64); if error != nil { return None() }; return Some(value); case \"Float.tryParse\": value, error := strconv.ParseFloat(String(args[0]), 64); if error != nil { return None() }; return Some(value); case \"Bool.tryParse\": value, error := strconv.ParseBool(String(args[0])); if error != nil { return None() }; return Some(value); case \"Int.toString\", \"Float.toString\", \"Bool.toString\": return Format(args[0]); case \"String.replace\": return strings.ReplaceAll(String(args[2]), String(args[0]), String(args[1])); case \"String.indexOf\": index := strings.Index(String(args[1]), String(args[0])); if index < 0 { return None() }; return Some(int64(index)); case \"String.toLower\": return strings.ToLower(String(args[0])); case \"String.toUpper\": return strings.ToUpper(String(args[0])); case \"String.substring\": start, length, value := Int(args[0]), Int(args[1]), String(args[2]); if start < 0 || length < 0 || start + length > int64(len(value)) { return None() }; return Some(value[start:start+length]); case \"String.concat\": values := List(args[1]); parts := make([]string, len(values)); for index, value := range values { parts[index] = String(value) }; return strings.Join(parts, String(args[0])); case \"String.split\": values := strings.Split(String(args[1]), String(args[0])); result := make([]any, len(values)); for index, value := range values { result[index] = value }; return result; case \"String.endsWith\": return strings.HasSuffix(String(args[1]), String(args[0])); case \"List.length\": return int64(len(List(args[0]))); case \"List.rev\": values := List(args[0]); result := make([]any, len(values)); for index, value := range values { result[len(values)-index-1] = value }; return result; case \"List.tryHead\": values := List(args[0]); if len(values) == 0 { return None() }; return Some(values[0]); case \"List.tail\": return Tail(args[0]); case \"List.append\": return Append(List(args[0]), List(args[1])); case \"List.contains\": for _, value := range List(args[1]) { if Equal(args[0], value) { return true } }; return false; case \"Option.isNone\": return !IsSome(args[0]); case \"Option.isSome\": return IsSome(args[0]); case \"Option.defaultValue\": option := AsOption(args[1]); if option.HasValue { return option.Value }; return args[0]; case \"Map.count\": return int64(len(AsMap(args[0]).Entries)); case \"Map.containsKey\": return IsSome(Index(args[1], args[0])); case \"Map.tryGet\": return Index(args[1], args[0]); default: panic(\"Unsupported FScript intrinsic: \" + name) } }\nfunc Format(value any) string { switch item := value.(type) { case bool: if item { return \"true\" }; return \"false\"; case Option: if !item.HasValue { return \"None\" }; return \"Some \" + Format(item.Value); case Union: if !item.HasValue { return item.CaseName }; return item.CaseName + \" \" + Format(item.Value); case OrderedMap: parts := make([]string, len(item.Entries)); for index, entry := range item.Entries { parts[index] = \"[\" + Format(entry.Key) + \"] = \" + Format(entry.Value) }; return \"{ \" + strings.Join(parts, \"; \") + \" }\"; case []any: parts := make([]string, len(item)); for index, value := range item { parts[index] = Format(value) }; return \"[\" + strings.Join(parts, \";\") + \"]\"; default: return fmt.Sprint(value) } }\nfunc Interpolate(values []any) string { var result strings.Builder; for _, value := range values { result.WriteString(Format(value)) }; return result.String() }\nfunc Fail(value any) { panic(fmt.Sprint(value)) }\n" } ] with ex -> diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs index cc74327..5de08e2 100644 --- a/src/FScript.Compiler/Hir.fs +++ b/src/FScript.Compiler/Hir.fs @@ -61,6 +61,8 @@ module Hir = | Map of MapEntry list | Intrinsic of string | HostExternal of string * int + | ForEach of SymbolId * string * Type * Expression * Expression + | TypeToken of Type and MapEntry = | KeyValue of Expression * Expression @@ -179,6 +181,9 @@ module Hir = entries |> List.iter (function KeyValue(key, value) -> expression (indent + 2) key; expression (indent + 2) value | Spread value -> expression (indent + 2) value) | Intrinsic name -> lines.Add(typed $"intrinsic {name}") | HostExternal(name, arity) -> lines.Add(typed $"host-extern {name}/{arity}") + | ForEach(id, name, _, source, body) -> + lines.Add(typed (sprintf "for %%%d %s" (symbolNumber id) name)); expression (indent + 2) source; expression (indent + 2) body + | TypeToken tokenType -> lines.Add(typed $"typeof {formatType tokenType}") for declaration in program.Declarations do lines.Add(sprintf "decl %%%d %s: %s" (symbolNumber declaration.Id) declaration.Name (formatType (let (Forall(_, value)) = declaration.Scheme in value))) diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index 98e32c4..4047c11 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -7,6 +7,7 @@ module HirLowering = { mutable NextSymbol: int ExpressionTypes: Map mutable UnionCases: Map + mutable TypeDefinitions: Map mutable Diagnostics: CompilerDiagnostic list } let private newSymbol state = @@ -205,6 +206,21 @@ module HirLowering = | Error a, Error b -> Error(a @ b) | Error errors, _ | _, Error errors -> Error errors | ERaise(value, _) -> lowerExpression state environment TString value |> Result.map (Hir.Fail >> make) + | ELetPattern(pattern, binding, body, _) -> + let bindingType = expressionType state binding (Types.freshVar()) + match lowerExpression state environment bindingType binding, lowerPattern state bindingType pattern with + | Ok loweredBinding, Ok(loweredPattern, bindings) -> + let bodyEnvironment = Map.fold (fun env name id -> Map.add name id env) environment bindings + lowerExpression state bodyEnvironment inferred body + |> Result.map (fun loweredBody -> make (Hir.Match(loweredBinding, [ { Pattern = loweredPattern; Guard = None; Body = loweredBody; Span = Ast.spanOfPattern pattern } ]))) + | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e + | EFor(name, source, body, _) -> + let sourceType = expressionType state source (TList(Types.freshVar())) + let itemType = match sourceType with TList item -> item | _ -> Types.freshVar() + let id = newSymbol state + match lowerExpression state environment sourceType source, lowerExpression state (environment |> Map.add name id) TUnit body with + | Ok loweredSource, Ok loweredBody -> Ok(make (Hir.ForEach(id, name, itemType, loweredSource, loweredBody))) + | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e | EMatch(target, cases, _) -> let targetType = expressionType state target (Types.freshVar()) match lowerExpression state environment targetType target with @@ -235,6 +251,10 @@ module HirLowering = | Ok loweredTarget, Ok loweredIndex -> Ok(make (Hir.IndexGet(loweredTarget, loweredIndex))) | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e | ENameOf(name, _) -> Ok(make (Hir.Constant(Hir.String name))) + | ETypeOf(name, _) -> + match state.TypeDefinitions |> Map.tryFind name with + | Some value -> Ok(make (Hir.TypeToken value)) + | None -> Error [ Diagnostic.hir "FSH1005" $"Unknown type '{name}'." (Some span) ] | EInterpolatedString(parts, _) -> parts |> List.map (function | IPText text -> Ok(Hir.TextPart text) @@ -255,7 +275,7 @@ module HirLowering = let expressionTypes = resolved.ExpressionTypes |> List.fold (fun state item -> state |> Map.add item.Expr item.Type) Map.empty - let state = { NextSymbol = 0; ExpressionTypes = expressionTypes; UnionCases = Map.empty; Diagnostics = [] } + let state = { NextSymbol = 0; ExpressionTypes = expressionTypes; UnionCases = Map.empty; TypeDefinitions = Map.empty; Diagnostics = [] } let builtins = BuiltinSignatures.builtinSchemes |> Map.toList @@ -279,6 +299,13 @@ module HirLowering = typeDefinitions |> List.filter (fun definition -> not definition.Cases.IsEmpty) |> List.map (fun definition -> let cases = definition.Cases |> List.map (fun (name, payload) -> name, Option.map typeFromRef payload) |> Map.ofList definition, TUnion(definition.Name, cases)) + state.TypeDefinitions <- + typeDefinitions |> List.map (fun definition -> + let value = + if not definition.Cases.IsEmpty then + TUnion(definition.Name, definition.Cases |> List.map (fun (name, payload) -> name, Option.map typeFromRef payload) |> Map.ofList) + else TRecord(definition.Fields |> List.map (fun (name, fieldType) -> name, typeFromRef fieldType) |> Map.ofList) + definition.Name, value) |> Map.ofList state.UnionCases <- unionDefinitions |> List.collect (fun (definition, _) -> let cases = definition.Cases |> List.map (fun (name, payload) -> name, Option.map typeFromRef payload) |> Map.ofList diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index 7c06891..c4fe91d 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -50,6 +50,8 @@ module HirValidation = entries |> List.map (function Hir.KeyValue(key, value) -> Set.union (references key) (references value) | Hir.Spread value -> references value) |> Set.unionMany | Hir.Intrinsic _ -> Set.empty | Hir.HostExternal _ -> Set.empty + | Hir.ForEach(id, _, _, source, body) -> Set.union (references source) (references body |> Set.remove id) + | Hir.TypeToken _ -> Set.empty let declared = declarationIds |> Set.ofList let unresolved = [ yield! program.Declarations |> List.collect (fun value -> references value.Value |> Set.toList) diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index d40e6c5..7a0af4c 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -186,6 +186,11 @@ module IrArtifact = writer.WriteEndArray() | Lir.Intrinsic name -> writer.WriteString("kind", "intrinsic"); writer.WriteString("name", name) | Lir.HostExternal(name, arity) -> writer.WriteString("kind", "host-external"); writer.WriteString("name", name); writer.WriteNumber("arity", arity) + | Lir.ForEach(id, name, itemType, source, body) -> + writer.WriteString("kind", "for-each"); writer.WriteNumber("symbol", Lir.symbolNumber id); writer.WriteString("name", name) + writer.WritePropertyName("itemType"); writeType writer itemType; writer.WritePropertyName("source"); writeExpression writer source + writer.WritePropertyName("body"); writeExpression writer body + | Lir.TypeToken tokenType -> writer.WriteString("kind", "type-token"); writer.WritePropertyName("tokenType"); writeType writer tokenType writer.WriteEndObject() let serialize (program: Lir.Program) = @@ -319,6 +324,9 @@ module IrArtifact = | kind -> invalidOp $"Unknown map entry '{kind}'.") |> Seq.toList) | "intrinsic" -> Lir.Intrinsic(stringProperty "name" element) | "host-external" -> Lir.HostExternal(stringProperty "name" element, (property "arity" element).GetInt32()) + | "for-each" -> Lir.ForEach(Lir.SymbolId((property "symbol" element).GetInt32()), stringProperty "name" element, + readType (property "itemType" element), readExpression (property "source" element), readExpression (property "body" element)) + | "type-token" -> Lir.TypeToken(readType (property "tokenType" element)) | kind -> invalidOp $"Unknown LIR expression kind '{kind}'." { Type = expressionType; Span = span; Node = node } diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index 31c886a..6cfa15f 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -49,6 +49,8 @@ module Lir = | Map of MapEntry list | Intrinsic of string | HostExternal of string * int + | ForEach of SymbolId * string * Type * Expression * Expression + | TypeToken of Type and MapEntry = | KeyValue of Expression * Expression @@ -162,6 +164,8 @@ module LirLowering = | Hir.Map entries -> Lir.Map(entries |> List.map (function Hir.KeyValue(key, item) -> Lir.KeyValue(expression key, expression item) | Hir.Spread item -> Lir.Spread(expression item))) | Hir.Intrinsic name -> Lir.Intrinsic name | Hir.HostExternal(name, arity) -> Lir.HostExternal(name, arity) + | Hir.ForEach(id, name, itemType, source, body) -> Lir.ForEach(symbol id, name, itemType, expression source, expression body) + | Hir.TypeToken tokenType -> Lir.TypeToken tokenType { Type = value.Type; Span = value.Span; Node = node } let lower (program: Hir.Program) : Lir.Program = diff --git a/src/FScript.Interpreter.Lir/LirInterpreter.fs b/src/FScript.Interpreter.Lir/LirInterpreter.fs index a485d03..2481855 100644 --- a/src/FScript.Interpreter.Lir/LirInterpreter.fs +++ b/src/FScript.Interpreter.Lir/LirInterpreter.fs @@ -20,6 +20,7 @@ type LirValue = | VMap of (LirValue * LirValue) list | VIntrinsic of string * LirValue list | VExternal of string * int * LirValue list + | VTypeToken of Type | VClosure of Lir.SymbolId * string * Lir.Expression * Map ref type ExecutionResult = { Value: LirValue } @@ -42,6 +43,7 @@ module LirInterpreter = | VOption(Some a), VOption(Some b) -> equal a b | VUnion(aType, aCase, aValue), VUnion(bType, bCase, bValue) -> aType = bType && aCase = bCase && equal (VOption aValue) (VOption bValue) | VMap a, VMap b -> a.Length = b.Length && List.forall2 (fun (ak, av) (bk, bv) -> equal ak bk && equal av bv) a b + | VTypeToken a, VTypeToken b -> a = b | _ -> false let rec private formatValue = function @@ -59,6 +61,7 @@ module LirInterpreter = | VClosure _ -> "" | VIntrinsic _ -> "" | VExternal _ -> "" + | VTypeToken value -> $"" let private fail code span message = Error [ { Code = code; Stage = Interpreter; Message = message; Span = Some span } ] @@ -286,6 +289,17 @@ module LirInterpreter = build [] entries | Lir.Intrinsic name -> Ok(VIntrinsic(name, [])) | Lir.HostExternal(name, arity) -> Ok(VExternal(name, arity, [])) + | Lir.ForEach(id, _, _, source, body) -> + match evaluate environment source with + | Ok(VList values) -> + let rec loop remaining = + match remaining with + | [] -> Ok VUnit + | value :: tail -> evaluate (environment |> Map.add id value) body |> Result.bind (fun _ -> loop tail) + loop values + | Ok _ -> fail "FSI3018" expression.Span "For-loop source must be a list." + | Error errors -> Error errors + | Lir.TypeToken tokenType -> Ok(VTypeToken tokenType) and private evaluateMany environment items = items |> List.map (evaluate environment) |> collect diff --git a/tests/FScript.Compiler.Tests/GoBackendTests.fs b/tests/FScript.Compiler.Tests/GoBackendTests.fs index 09d6cfd..f50cd6d 100644 --- a/tests/FScript.Compiler.Tests/GoBackendTests.fs +++ b/tests/FScript.Compiler.Tests/GoBackendTests.fs @@ -25,6 +25,9 @@ type GoBackendTests() = [] [] [] + [] + [] + [] member _.``Generated projects run with Go`` (source: string, expected: string) = let input = { Program = Parser.parseProgram source; Externs = [] } let compiled = diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index be56692..554f11a 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -153,3 +153,17 @@ type HirTests() = match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok _ -> Assert.Fail("Expected unsupported external diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSI3021")) + + [] + member _.``Executes local binding patterns iteration and type tokens`` () = + let cases = + [ "let sum pair = (let (left, right) = pair\n left + right\n)\nsum (1, 2)", LirValue.VInt 3L + "for value in [1; 2; 3] do ignore value", LirValue.VUnit + "type Package = { Name: string }\ntypeof Package", LirValue.VTypeToken(TRecord(Map.ofList [ "Name", TString ])) ] + for source, expected in cases do + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(expected), source) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index 75d594e..562db41 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -49,7 +49,10 @@ type IrArtifactTests() = "([1..3][0], $\"answer {42}\")" "let value = {| Count = 1 |}\n{| value with Count = 2 |}" "type Shape = | Point | Circle of int\nmatch Circle 2 with\n | Point -> 0\n | Circle radius -> radius" - "{ [\"answer\"] = 42 }[\"answer\"]" ] do + "{ [\"answer\"] = 42 }[\"answer\"]" + "let sum pair = (let (left, right) = pair\n left + right\n)\nsum (1, 2)" + "for value in [1; 2; 3] do ignore value" + "type Package = { Name: string }\ntypeof Package" ] do let bytes = compile source |> _.Lir |> IrArtifact.serialize match IrArtifact.deserialize bytes with | Ok _ -> () From 6a0dc4f650dd7a819c8d1e75f8a726cfff19fbdf Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 17:28:38 +0200 Subject: [PATCH 019/133] test(compiler): isolate backend consumer suites --- CHANGELOG.md | 1 + FScript.sln | 30 +++ docs/architecture/assemblies-and-roles.md | 2 + docs/architecture/typed-ir-and-backends.md | 2 + src/FScript.Interpreter.Lir/LirInterpreter.fs | 97 ++++++++-- .../FScript.Backend.Go.Tests.fsproj | 20 ++ .../GoBackendTests.fs | 2 +- tests/FScript.Backend.Go.Tests/Program.fs | 4 + .../FScript.Compiler.Tests.fsproj | 3 - tests/FScript.Compiler.Tests/HirTests.fs | 149 --------------- .../FScript.Compiler.Tests/IrArtifactTests.fs | 7 +- .../FScript.Interpreter.Lir.Tests.fsproj | 20 ++ .../LirInterpreterTests.fs | 174 ++++++++++++++++++ .../FScript.Interpreter.Lir.Tests/Program.fs | 4 + 14 files changed, 345 insertions(+), 170 deletions(-) create mode 100644 tests/FScript.Backend.Go.Tests/FScript.Backend.Go.Tests.fsproj rename tests/{FScript.Compiler.Tests => FScript.Backend.Go.Tests}/GoBackendTests.fs (98%) create mode 100644 tests/FScript.Backend.Go.Tests/Program.fs create mode 100644 tests/FScript.Interpreter.Lir.Tests/FScript.Interpreter.Lir.Tests.fsproj create mode 100644 tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs create mode 100644 tests/FScript.Interpreter.Lir.Tests/Program.fs diff --git a/CHANGELOG.md b/CHANGELOG.md index 9ed365a..d51ecf0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added dedicated LIR-interpreter and Go-backend test assemblies, and completed higher-order collection intrinsics in LIR execution. - Added compiled local binding patterns, sequential list iteration, and record/union type tokens across LIR artifacts, interpretation, and Go generation. - Split LIR execution and Go generation into independent `FScript.Interpreter.Lir` and `FScript.Backend.Go` assemblies. - Added serializable host-extern linkage that remains inert when unused and reports a dedicated LIR diagnostic when reached. diff --git a/FScript.sln b/FScript.sln index 6c66187..07d3c6f 100644 --- a/FScript.sln +++ b/FScript.sln @@ -39,6 +39,10 @@ Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "FScript.Interpreter.Lir", " EndProject Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "FScript.Backend.Go", "src\FScript.Backend.Go\FScript.Backend.Go.fsproj", "{FC0FF53F-A10D-40B2-9C30-87AF93AD8A52}" EndProject +Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "FScript.Interpreter.Lir.Tests", "tests\FScript.Interpreter.Lir.Tests\FScript.Interpreter.Lir.Tests.fsproj", "{B9A3B895-A44F-4D1F-BFB9-4E1567885E64}" +EndProject +Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "FScript.Backend.Go.Tests", "tests\FScript.Backend.Go.Tests\FScript.Backend.Go.Tests.fsproj", "{966D205D-F585-4294-9486-72F317ADE7D4}" +EndProject Global GlobalSection(SolutionConfigurationPlatforms) = preSolution Debug|Any CPU = Debug|Any CPU @@ -241,6 +245,30 @@ Global {FC0FF53F-A10D-40B2-9C30-87AF93AD8A52}.Release|x64.Build.0 = Release|Any CPU {FC0FF53F-A10D-40B2-9C30-87AF93AD8A52}.Release|x86.ActiveCfg = Release|Any CPU {FC0FF53F-A10D-40B2-9C30-87AF93AD8A52}.Release|x86.Build.0 = Release|Any CPU + {B9A3B895-A44F-4D1F-BFB9-4E1567885E64}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {B9A3B895-A44F-4D1F-BFB9-4E1567885E64}.Debug|Any CPU.Build.0 = Debug|Any CPU + {B9A3B895-A44F-4D1F-BFB9-4E1567885E64}.Debug|x64.ActiveCfg = Debug|Any CPU + {B9A3B895-A44F-4D1F-BFB9-4E1567885E64}.Debug|x64.Build.0 = Debug|Any CPU + {B9A3B895-A44F-4D1F-BFB9-4E1567885E64}.Debug|x86.ActiveCfg = Debug|Any CPU + {B9A3B895-A44F-4D1F-BFB9-4E1567885E64}.Debug|x86.Build.0 = Debug|Any CPU + {B9A3B895-A44F-4D1F-BFB9-4E1567885E64}.Release|Any CPU.ActiveCfg = Release|Any CPU + {B9A3B895-A44F-4D1F-BFB9-4E1567885E64}.Release|Any CPU.Build.0 = Release|Any CPU + {B9A3B895-A44F-4D1F-BFB9-4E1567885E64}.Release|x64.ActiveCfg = Release|Any CPU + {B9A3B895-A44F-4D1F-BFB9-4E1567885E64}.Release|x64.Build.0 = Release|Any CPU + {B9A3B895-A44F-4D1F-BFB9-4E1567885E64}.Release|x86.ActiveCfg = Release|Any CPU + {B9A3B895-A44F-4D1F-BFB9-4E1567885E64}.Release|x86.Build.0 = Release|Any CPU + {966D205D-F585-4294-9486-72F317ADE7D4}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {966D205D-F585-4294-9486-72F317ADE7D4}.Debug|Any CPU.Build.0 = Debug|Any CPU + {966D205D-F585-4294-9486-72F317ADE7D4}.Debug|x64.ActiveCfg = Debug|Any CPU + {966D205D-F585-4294-9486-72F317ADE7D4}.Debug|x64.Build.0 = Debug|Any CPU + {966D205D-F585-4294-9486-72F317ADE7D4}.Debug|x86.ActiveCfg = Debug|Any CPU + {966D205D-F585-4294-9486-72F317ADE7D4}.Debug|x86.Build.0 = Debug|Any CPU + {966D205D-F585-4294-9486-72F317ADE7D4}.Release|Any CPU.ActiveCfg = Release|Any CPU + {966D205D-F585-4294-9486-72F317ADE7D4}.Release|Any CPU.Build.0 = Release|Any CPU + {966D205D-F585-4294-9486-72F317ADE7D4}.Release|x64.ActiveCfg = Release|Any CPU + {966D205D-F585-4294-9486-72F317ADE7D4}.Release|x64.Build.0 = Release|Any CPU + {966D205D-F585-4294-9486-72F317ADE7D4}.Release|x86.ActiveCfg = Release|Any CPU + {966D205D-F585-4294-9486-72F317ADE7D4}.Release|x86.Build.0 = Release|Any CPU EndGlobalSection GlobalSection(SolutionProperties) = preSolution HideSolutionNode = FALSE @@ -262,5 +290,7 @@ Global {CEB16446-D96F-40A7-9771-F90B9A78D7B5} = {827E0CD3-B72D-47B6-A68D-7590B98EB39B} {40BB6F66-433C-407A-B71B-AF8A7DD2992A} = {827E0CD3-B72D-47B6-A68D-7590B98EB39B} {FC0FF53F-A10D-40B2-9C30-87AF93AD8A52} = {827E0CD3-B72D-47B6-A68D-7590B98EB39B} + {B9A3B895-A44F-4D1F-BFB9-4E1567885E64} = {0AB3BF05-4346-4AA6-1389-037BE0695223} + {966D205D-F585-4294-9486-72F317ADE7D4} = {0AB3BF05-4346-4AA6-1389-037BE0695223} EndGlobalSection EndGlobal diff --git a/docs/architecture/assemblies-and-roles.md b/docs/architecture/assemblies-and-roles.md index 8956b88..11b521a 100644 --- a/docs/architecture/assemblies-and-roles.md +++ b/docs/architecture/assemblies-and-roles.md @@ -71,6 +71,8 @@ Responsibilities: `FScript.Backend.Go` depends on `FScript.Compiler`; the core compiler has no dependency on the Go backend or a Go installation. +Consumer tests follow the same dependency boundary: `FScript.Interpreter.Lir.Tests` owns executable LIR conformance and `FScript.Backend.Go.Tests` owns generated-project integration. `FScript.Compiler.Tests` references neither consumer and covers HIR, lowering, LIR validation, and artifacts only. + NuGet: - `MagnusOpera.FScript.Language` diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 7f9386b..6ae056a 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -30,6 +30,8 @@ Top-level mutual-recursive declarations are initialized as an atomic closure gro Language builtins are explicit `Intrinsic` declarations rather than magic unresolved references. Consumers accumulate their curried arguments according to the catalog signature and dispatch only at full application. The initial portable implementations cover scalar parsing/formatting, strings, simple list and option operations, and map queries; a reachable catalog entry without an implementation receives a stage-specific unsupported diagnostic or runtime failure. +The LIR interpreter implements the pure builtin catalog, including callbacks for list mapping, selection, collection, folding, filtering and search; lazy option fallback and mapping; and ordered-map construction, folds, transforms, iteration, and removal. Callback application goes through the same curried closure path as ordinary source calls, preserving failure propagation and evaluation order. + Host externs are copied into IR as name, signature, and arity only; their executable .NET delegates never cross the compiler boundary. Constructing or retaining an unused extern handle is inert. Fully applying one in the LIR interpreter produces `FSI3021`, preserving the rule that unreachable unsupported linkage does not block execution. Local binding patterns reuse the same single-scrutinee decision-tree representation as `match`. Sequential `for ... in ... do ...` is explicit in HIR and LIR so each consumer evaluates the input list once, visits values in order, and discards each body result before returning unit. `typeof` resolves declared record and union definitions during HIR construction and carries the semantic type as a serializable type-token node. diff --git a/src/FScript.Interpreter.Lir/LirInterpreter.fs b/src/FScript.Interpreter.Lir/LirInterpreter.fs index 2481855..50bd019 100644 --- a/src/FScript.Interpreter.Lir/LirInterpreter.fs +++ b/src/FScript.Interpreter.Lir/LirInterpreter.fs @@ -71,8 +71,21 @@ module LirInterpreter = let rec count = function TFun(_, result) -> 1 + count result | _ -> 0 count valueType - let private invokeIntrinsic span name arguments = + let private invokeIntrinsic apply span name arguments = let unsupported () = fail "FSI3020" span $"Intrinsic '{name}' is not yet executable in LIR." + let expectBool = function Ok(VBool value) -> Ok value | Ok _ -> fail "FSI3022" span $"Intrinsic '{name}' callback must return bool." | Error errors -> Error errors + let rec mapCallback callback acc = function + | [] -> Ok(List.rev acc) + | value :: rest -> apply callback value |> Result.bind (fun mapped -> mapCallback callback (mapped :: acc) rest) + let rec filterCallback callback acc = function + | [] -> Ok(List.rev acc) + | value :: rest -> apply callback value |> expectBool |> Result.bind (fun keep -> filterCallback callback (if keep then value :: acc else acc) rest) + let mapSet key value entries = + let rec loop acc = function + | [] -> List.rev ((key, value) :: acc) + | (existing, _) :: rest when equal existing key -> List.rev acc @ ((key, value) :: rest) + | entry :: rest -> loop (entry :: acc) rest + loop [] entries match name, arguments with | "ignore", [ _ ] -> Ok VUnit | "Int.tryParse", [ VString value ] -> match System.Int64.TryParse value with true, parsed -> Ok(VOption(Some(VInt parsed))) | _ -> Ok(VOption None) @@ -91,7 +104,36 @@ module LirInterpreter = | "String.split", [ VString separator; VString value ] -> value.Split([| separator |], System.StringSplitOptions.None) |> Array.toList |> List.map VString |> VList |> Ok | "String.endsWith", [ VString suffix; VString value ] -> Ok(VBool(value.EndsWith(suffix, System.StringComparison.Ordinal))) | "List.length", [ VList values ] -> Ok(VInt(int64 values.Length)) + | "List.empty", [] -> Ok(VList []) + | "List.map", [ callback; VList values ] -> mapCallback callback [] values |> Result.map VList + | "List.iter", [ callback; VList values ] -> mapCallback callback [] values |> Result.map (fun _ -> VUnit) + | "List.choose", [ callback; VList values ] -> + mapCallback callback [] values |> Result.bind (fun mapped -> + mapped |> List.fold (fun state value -> + match state, value with + | Ok acc, VOption(Some item) -> Ok(item :: acc) + | Ok acc, VOption None -> Ok acc + | Ok _, _ -> fail "FSI3022" span "List.choose callback must return option." + | Error errors, _ -> Error errors) (Ok []) |> Result.map (List.rev >> VList)) + | "List.collect", [ callback; VList values ] -> + mapCallback callback [] values |> Result.bind (fun mapped -> + mapped |> List.fold (fun state value -> + match state, value with Ok acc, VList items -> Ok(acc @ items) | Ok _, _ -> fail "FSI3022" span "List.collect callback must return list." | Error errors, _ -> Error errors) (Ok []) |> Result.map VList) + | "List.exists", [ callback; VList values ] -> + let rec loop = function [] -> Ok(VBool false) | value :: rest -> apply callback value |> expectBool |> Result.bind (fun found -> if found then Ok(VBool true) else loop rest) + loop values | "List.rev", [ VList values ] -> Ok(VList(List.rev values)) + | "List.distinct", [ VList values ] -> values |> List.fold (fun acc value -> if List.exists (equal value) acc then acc else acc @ [ value ]) [] |> VList |> Ok + | "List.fold", [ callback; state; VList values ] -> + values |> List.fold (fun result value -> result |> Result.bind (fun current -> apply callback current |> Result.bind (fun partial -> apply partial value))) (Ok state) + | "List.filter", [ callback; VList values ] -> filterCallback callback [] values |> Result.map VList + | "List.tryFind", [ callback; VList values ] -> + let rec loop = function [] -> Ok(VOption None) | value :: rest -> apply callback value |> expectBool |> Result.bind (fun found -> if found then Ok(VOption(Some value)) else loop rest) + loop values + | "List.tryGet", [ callback; VList values ] -> + let rec loop index = function [] -> Ok(VOption None) | value :: rest -> apply callback value |> expectBool |> Result.bind (fun found -> if found then Ok(VOption(Some(VInt index))) else loop (index + 1L) rest) + loop 0L values + | "List.tryItem", [ VInt index; VList values ] -> if index < 0L || index >= int64 values.Length then Ok(VOption None) else Ok(VOption(Some values.[int index])) | "List.tryHead", [ VList values ] -> Ok(VOption(List.tryHead values)) | "List.tail", [ VList(_ :: tail) ] -> Ok(VList tail) | "List.append", [ VList left; VList right ] -> Ok(VList(left @ right)) @@ -99,9 +141,34 @@ module LirInterpreter = | "Option.isNone", [ VOption value ] -> Ok(VBool value.IsNone) | "Option.isSome", [ VOption value ] -> Ok(VBool value.IsSome) | "Option.defaultValue", [ fallback; VOption value ] -> Ok(Option.defaultValue fallback value) + | "Option.defaultWith", [ fallback; VOption(Some value) ] -> Ok value + | "Option.defaultWith", [ fallback; VOption None ] -> apply fallback VUnit + | "Option.map", [ callback; VOption(Some value) ] -> apply callback value |> Result.map (Some >> VOption) + | "Option.map", [ _; VOption None ] -> Ok(VOption None) + | "Map.empty", [] -> Ok(VMap []) | "Map.count", [ VMap entries ] -> Ok(VInt(int64 entries.Length)) | "Map.containsKey", [ key; VMap entries ] -> Ok(VBool(entries |> List.exists (fst >> equal key))) | "Map.tryGet", [ key; VMap entries ] -> Ok(VOption(entries |> List.tryPick (fun (itemKey, value) -> if equal itemKey key then Some value else None))) + | "Map.add", [ key; value; VMap entries ] -> Ok(VMap(mapSet key value entries)) + | "Map.ofList", [ VList pairs ] -> + pairs |> List.fold (fun state pair -> match state, pair with Ok entries, VTuple [ key; value ] -> Ok(mapSet key value entries) | Ok _, _ -> fail "FSI3022" span "Map.ofList expects key/value tuples." | Error errors, _ -> Error errors) (Ok []) |> Result.map VMap + | "Map.fold", [ callback; state; VMap entries ] -> + entries |> List.fold (fun result (key, value) -> result |> Result.bind (fun current -> apply callback current |> Result.bind (fun withKey -> apply withKey key |> Result.bind (fun withValue -> apply withValue value)))) (Ok state) + | "Map.filter", [ callback; VMap entries ] -> + let rec loop acc = function [] -> Ok(VMap(List.rev acc)) | (key, value as entry) :: rest -> apply callback key |> Result.bind (fun partial -> apply partial value |> expectBool |> Result.bind (fun keep -> loop (if keep then entry :: acc else acc) rest)) + loop [] entries + | "Map.choose", [ callback; VMap entries ] -> + let rec loop acc = function + | [] -> Ok(VMap(List.rev acc)) + | (key, value) :: rest -> apply callback key |> Result.bind (fun partial -> apply partial value |> Result.bind (function VOption(Some chosen) -> loop ((key, chosen) :: acc) rest | VOption None -> loop acc rest | _ -> fail "FSI3022" span "Map.choose callback must return option.")) + loop [] entries + | "Map.map", [ callback; VMap entries ] -> + let rec loop acc = function [] -> Ok(VMap(List.rev acc)) | (key, value) :: rest -> apply callback value |> Result.bind (fun mapped -> loop ((key, mapped) :: acc) rest) + loop [] entries + | "Map.iter", [ callback; VMap entries ] -> + let rec loop = function [] -> Ok VUnit | (key, value) :: rest -> apply callback key |> Result.bind (fun partial -> apply partial value |> Result.bind (fun _ -> loop rest)) + loop entries + | "Map.remove", [ key; VMap entries ] -> Ok(VMap(entries |> List.filter (fst >> equal key >> not))) | _ -> unsupported () let rec private matchPattern (pattern: Lir.Pattern) value = @@ -145,16 +212,7 @@ module LirInterpreter = | Lir.MakeClosure(id, name, _, body) -> Ok(VClosure(id, name, body, ref environment)) | Lir.CallClosure(fn, argument) -> match evaluate environment fn, evaluate environment argument with - | Ok(VClosure(parameter, _, body, captured)), Ok value -> evaluate (captured.Value |> Map.add parameter value) body - | Ok(VIntrinsic(name, existing)), Ok value -> - let arguments = existing @ [ value ] - if arguments.Length = intrinsicArity name then invokeIntrinsic expression.Span name arguments - else Ok(VIntrinsic(name, arguments)) - | Ok(VExternal(name, arity, existing)), Ok value -> - let arguments = existing @ [ value ] - if arguments.Length = arity then fail "FSI3021" expression.Span $"Host external '{name}' is not executable from LIR." - else Ok(VExternal(name, arity, arguments)) - | Ok _, Ok _ -> fail "FSI3003" expression.Span "Attempted to call a non-function value." + | Ok fnValue, Ok value -> applyValue expression.Span fnValue value | Error a, Error b -> Error(a @ b) | Error errors, _ | _, Error errors -> Error errors | Lir.Branch(condition, whenTrue, whenFalse) -> @@ -287,7 +345,9 @@ module LirInterpreter = | Ok _ -> fail "FSI3017" expression.Span "Map spread requires a map." | Error errors -> Error errors build [] entries - | Lir.Intrinsic name -> Ok(VIntrinsic(name, [])) + | Lir.Intrinsic name -> + if intrinsicArity name = 0 then invokeIntrinsic (applyValue expression.Span) expression.Span name [] + else Ok(VIntrinsic(name, [])) | Lir.HostExternal(name, arity) -> Ok(VExternal(name, arity, [])) | Lir.ForEach(id, _, _, source, body) -> match evaluate environment source with @@ -301,6 +361,19 @@ module LirInterpreter = | Error errors -> Error errors | Lir.TypeToken tokenType -> Ok(VTypeToken tokenType) + and private applyValue span fn value = + match fn with + | VClosure(parameter, _, body, captured) -> evaluate (captured.Value |> Map.add parameter value) body + | VIntrinsic(name, existing) -> + let arguments = existing @ [ value ] + if arguments.Length = intrinsicArity name then invokeIntrinsic (applyValue span) span name arguments + else Ok(VIntrinsic(name, arguments)) + | VExternal(name, arity, existing) -> + let arguments = existing @ [ value ] + if arguments.Length = arity then fail "FSI3021" span $"Host external '{name}' is not executable from LIR." + else Ok(VExternal(name, arity, arguments)) + | _ -> fail "FSI3003" span "Attempted to call a non-function value." + and private evaluateMany environment items = items |> List.map (evaluate environment) |> collect and private collect<'a> (results: Result<'a, CompilerDiagnostic list> list) : Result<'a list, CompilerDiagnostic list> = diff --git a/tests/FScript.Backend.Go.Tests/FScript.Backend.Go.Tests.fsproj b/tests/FScript.Backend.Go.Tests/FScript.Backend.Go.Tests.fsproj new file mode 100644 index 0000000..3aa17bf --- /dev/null +++ b/tests/FScript.Backend.Go.Tests/FScript.Backend.Go.Tests.fsproj @@ -0,0 +1,20 @@ + + + net10.0 + false + false + + + + + + + + + + + + + + + diff --git a/tests/FScript.Compiler.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs similarity index 98% rename from tests/FScript.Compiler.Tests/GoBackendTests.fs rename to tests/FScript.Backend.Go.Tests/GoBackendTests.fs index f50cd6d..8ec196e 100644 --- a/tests/FScript.Compiler.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -1,4 +1,4 @@ -namespace FScript.Compiler.Tests +namespace FScript.Backend.Go.Tests open System open System.Diagnostics diff --git a/tests/FScript.Backend.Go.Tests/Program.fs b/tests/FScript.Backend.Go.Tests/Program.fs new file mode 100644 index 0000000..f049cb4 --- /dev/null +++ b/tests/FScript.Backend.Go.Tests/Program.fs @@ -0,0 +1,4 @@ +module FScript.Backend.Go.Tests.Program + +[] +let main _ = 0 diff --git a/tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj b/tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj index 83a18fb..2edc418 100644 --- a/tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj +++ b/tests/FScript.Compiler.Tests/FScript.Compiler.Tests.fsproj @@ -7,7 +7,6 @@ - @@ -17,7 +16,5 @@ - - diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index 554f11a..d397ceb 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -18,152 +18,3 @@ type HirTests() = Assert.That(actual, Does.Contain("primitive IntAdd")) Assert.That(actual, Does.Contain("entry")) Assert.That(Hir.format compiled.Hir, Is.EqualTo(actual)) - match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with - | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 3L)) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - - [] - member _.``Executes aggregate expressions in LIR`` () = - let input = { Program = Parser.parseProgram "[1] @ [2; 3]"; Externs = [] } - match Compiler.compile input with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - | Ok compiled -> - match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with - | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VList [ LirValue.VInt 1L; LirValue.VInt 2L; LirValue.VInt 3L ])) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - - [] - [] - [] - member _.``Lowers tuples options and records`` (source: string) = - match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - | Ok compiled -> - match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with - | Ok _ -> Assert.Pass() - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - - [] - member _.``Executes lexical let bindings and recursive closures`` () = - let source = "let f x = (let rec loop n = if n = 0 then x else loop (n - 1)\n loop 2\n)\nf 7" - match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - | Ok compiled -> - match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with - | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 7L)) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - - [] - member _.``Raise preserves an interpreter diagnostic`` () = - match Compiler.compile { Program = Parser.parseProgram "raise \"boom\""; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - | Ok compiled -> - match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with - | Ok _ -> Assert.Fail("Expected failure") - | Error diagnostics -> - Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSI3011")) - Assert.That(diagnostics |> List.map _.Message, Does.Contain("boom")) - - [ x\n | [] -> 0", 1L)>] - [ x + 1\n | None -> 0", 4L)>] - [ x\n | _ -> 0", 1L)>] - [ x\n | _ -> 42", 42L)>] - member _.``Executes deterministic pattern decision trees`` (source: string, expected: int64) = - match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - | Ok compiled -> - match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with - | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt expected)) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - - [] - member _.``Executes ranges indexing updates and interpolation`` () = - let cases = - [ "[1..3]", LirValue.VList [ LirValue.VInt 1L; LirValue.VInt 2L; LirValue.VInt 3L ] - "[1; 2][1]", LirValue.VOption(Some(LirValue.VInt 2L)) - "let value = {| Count = 1 |}\n{| value with Count = 2 |}.Count", LirValue.VInt 2L - "$\"value {1 + 2}\"", LirValue.VString "value 3" ] - for source, expected in cases do - match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - | Ok compiled -> - match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with - | Ok result -> Assert.That(result.Value, Is.EqualTo(expected), source) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - - [] - member _.``Executes declared union construction and matching`` () = - let source = "type Shape = | Point | Circle of int\nlet radius shape =\n match shape with\n | Point -> 0\n | Circle r -> r\nradius (Circle 3)" - match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - | Ok compiled -> - match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with - | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 3L)) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - - [] - member _.``Executes ordered map construction spread and indexing`` () = - let source = "let tail = { [\"a\"] = 2; [\"b\"] = 3 }\nlet values = { [\"a\"] = 1; ..tail }\nvalues[\"b\"]" - match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - | Ok compiled -> - match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with - | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VOption(Some(LirValue.VInt 3L)))) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - - [] - member _.``Initializes mutual recursive groups atomically`` () = - let source = "let rec even n =\n if n = 0 then true else odd (n - 1)\nand odd n =\n if n = 0 then false else even (n - 1)\n\neven 10" - match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - | Ok compiled -> - match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with - | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VBool true)) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - - [] - [] - [] - [] - member _.``Executes portable scalar and collection intrinsics`` (source: string, expected: string) = - match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - | Ok compiled -> - match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with - | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VString expected)) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - - [] - member _.``Retains host linkage but rejects only reachable external calls`` () = - let external : ExternalFunction = - { Name = "Host.echo" - Scheme = Forall([], TFun(TString, TString)) - Arity = 1 - Impl = fun _ values -> values.Head } - let compile source = Compiler.compile { Program = Parser.parseProgram source; Externs = [ external ] } - match compile "1" with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - | Ok compiled -> - match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with - | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 1L)) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - match compile "Host.echo \"value\"" with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - | Ok compiled -> - match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with - | Ok _ -> Assert.Fail("Expected unsupported external diagnostic") - | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSI3021")) - - [] - member _.``Executes local binding patterns iteration and type tokens`` () = - let cases = - [ "let sum pair = (let (left, right) = pair\n left + right\n)\nsum (1, 2)", LirValue.VInt 3L - "for value in [1; 2; 3] do ignore value", LirValue.VUnit - "type Package = { Name: string }\ntypeof Package", LirValue.VTypeToken(TRecord(Map.ofList [ "Name", TString ])) ] - for source, expected in cases do - match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) - | Ok compiled -> - match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with - | Ok result -> Assert.That(result.Value, Is.EqualTo(expected), source) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index 562db41..b7d34c4 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -14,17 +14,14 @@ type IrArtifactTests() = | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) [] - member _.``IR artifact is deterministic and executable after round trip`` () = + member _.``IR artifact is deterministic and equivalent after round trip`` () = let compiled = compile "let add x y = x + y\n(Some (add 1 2), [1] @ [2])" let first = IrArtifact.serialize compiled.Lir let second = IrArtifact.serialize compiled.Lir Assert.That((first = second), Is.True) match IrArtifact.deserialize first with | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) - | Ok loaded -> - match LirInterpreter.execute { ScriptName = None; Arguments = [] } loaded with - | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VTuple [ LirValue.VOption(Some(LirValue.VInt 3L)); LirValue.VList [ LirValue.VInt 1L; LirValue.VInt 2L ] ])) - | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) + | Ok loaded -> Assert.That(loaded, Is.EqualTo(compiled.Lir)) [] member _.``IR artifact rejects incompatible versions`` () = diff --git a/tests/FScript.Interpreter.Lir.Tests/FScript.Interpreter.Lir.Tests.fsproj b/tests/FScript.Interpreter.Lir.Tests/FScript.Interpreter.Lir.Tests.fsproj new file mode 100644 index 0000000..652fa5f --- /dev/null +++ b/tests/FScript.Interpreter.Lir.Tests/FScript.Interpreter.Lir.Tests.fsproj @@ -0,0 +1,20 @@ + + + net10.0 + false + false + + + + + + + + + + + + + + + diff --git a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs new file mode 100644 index 0000000..a52fb0f --- /dev/null +++ b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs @@ -0,0 +1,174 @@ +namespace FScript.Interpreter.Lir.Tests + +open FScript.Compiler +open FScript.Language +open NUnit.Framework + +[] +type LirInterpreterTests() = + + [] + member _.``Executes aggregate expressions in LIR`` () = + let input = { Program = Parser.parseProgram "[1] @ [2; 3]"; Externs = [] } + match Compiler.compile input with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VList [ LirValue.VInt 1L; LirValue.VInt 2L; LirValue.VInt 3L ])) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + [] + [] + member _.``Lowers tuples options and records`` (source: string) = + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok _ -> Assert.Pass() + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``Executes lexical let bindings and recursive closures`` () = + let source = "let f x = (let rec loop n = if n = 0 then x else loop (n - 1)\n loop 2\n)\nf 7" + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 7L)) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``Raise preserves an interpreter diagnostic`` () = + match Compiler.compile { Program = Parser.parseProgram "raise \"boom\""; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok _ -> Assert.Fail("Expected failure") + | Error diagnostics -> + Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSI3011")) + Assert.That(diagnostics |> List.map _.Message, Does.Contain("boom")) + + [ x\n | [] -> 0", 1L)>] + [ x + 1\n | None -> 0", 4L)>] + [ x\n | _ -> 0", 1L)>] + [ x\n | _ -> 42", 42L)>] + member _.``Executes deterministic pattern decision trees`` (source: string, expected: int64) = + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt expected)) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``Executes ranges indexing updates and interpolation`` () = + let cases = + [ "[1..3]", LirValue.VList [ LirValue.VInt 1L; LirValue.VInt 2L; LirValue.VInt 3L ] + "[1; 2][1]", LirValue.VOption(Some(LirValue.VInt 2L)) + "let value = {| Count = 1 |}\n{| value with Count = 2 |}.Count", LirValue.VInt 2L + "$\"value {1 + 2}\"", LirValue.VString "value 3" ] + for source, expected in cases do + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(expected), source) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``Executes declared union construction and matching`` () = + let source = "type Shape = | Point | Circle of int\nlet radius shape =\n match shape with\n | Point -> 0\n | Circle r -> r\nradius (Circle 3)" + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 3L)) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``Executes ordered map construction spread and indexing`` () = + let source = "let tail = { [\"a\"] = 2; [\"b\"] = 3 }\nlet values = { [\"a\"] = 1; ..tail }\nvalues[\"b\"]" + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VOption(Some(LirValue.VInt 3L)))) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``Initializes mutual recursive groups atomically`` () = + let source = "let rec even n =\n if n = 0 then true else odd (n - 1)\nand odd n =\n if n = 0 then false else even (n - 1)\n\neven 10" + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VBool true)) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + [] + [] + [] + member _.``Executes portable scalar and collection intrinsics`` (source: string, expected: string) = + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VString expected)) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``Retains host linkage but rejects only reachable external calls`` () = + let external : ExternalFunction = + { Name = "Host.echo" + Scheme = Forall([], TFun(TString, TString)) + Arity = 1 + Impl = fun _ values -> values.Head } + let compile source = Compiler.compile { Program = Parser.parseProgram source; Externs = [ external ] } + match compile "1" with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 1L)) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + match compile "Host.echo \"value\"" with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok _ -> Assert.Fail("Expected unsupported external diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSI3021")) + + [] + member _.``Executes local binding patterns iteration and type tokens`` () = + let cases = + [ "let sum pair = (let (left, right) = pair\n left + right\n)\nsum (1, 2)", LirValue.VInt 3L + "for value in [1; 2; 3] do ignore value", LirValue.VUnit + "type Package = { Name: string }\ntypeof Package", LirValue.VTypeToken(TRecord(Map.ofList [ "Name", TString ])) ] + for source, expected in cases do + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(expected), source) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``Executes the complete portable collection intrinsic catalog`` () = + let cases = + [ "List.map (fun value -> value + 1) [1; 2]", LirValue.VList [ LirValue.VInt 2L; LirValue.VInt 3L ] + "List.choose (fun value -> if value > 1 then Some value else None) [1; 2; 3]", LirValue.VList [ LirValue.VInt 2L; LirValue.VInt 3L ] + "List.collect (fun value -> [value; value]) [1; 2]", LirValue.VList [ LirValue.VInt 1L; LirValue.VInt 1L; LirValue.VInt 2L; LirValue.VInt 2L ] + "List.fold (fun state value -> state + value) 0 [1; 2; 3]", LirValue.VInt 6L + "List.tryGet (fun value -> value = 2) [1; 2; 3]", LirValue.VOption(Some(LirValue.VInt 1L)) + "Option.map (fun value -> value + 1) (Some 4)", LirValue.VOption(Some(LirValue.VInt 5L)) + "Option.defaultWith (fun _ -> 7) None", LirValue.VInt 7L + "Map.map (fun value -> value + 1) (Map.filter (fun key value -> value > 1) (Map.ofList [(\"b\", 2); (\"a\", 1)]))", LirValue.VMap [ LirValue.VString "b", LirValue.VInt 3L ] + "Map.fold (fun state key value -> state + value) 0 { [\"a\"] = 1; [\"b\"] = 2 }", LirValue.VInt 3L ] + for source, expected in cases do + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(expected), source) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) diff --git a/tests/FScript.Interpreter.Lir.Tests/Program.fs b/tests/FScript.Interpreter.Lir.Tests/Program.fs new file mode 100644 index 0000000..9bce492 --- /dev/null +++ b/tests/FScript.Interpreter.Lir.Tests/Program.fs @@ -0,0 +1,4 @@ +module FScript.Interpreter.Lir.Tests.Program + +[] +let main _ = 0 From b81097e4074420e00a475da9e4ff04ea86263e82 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 17:32:07 +0200 Subject: [PATCH 020/133] feat(go): execute higher-order runtime intrinsics --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 + docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Backend.Go/GoBackend.fs | 54 ++++++++++++++++--- .../GoBackendTests.fs | 18 +++++++ 5 files changed, 68 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d51ecf0..a209d35 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added a universal erased Go closure convention and generated-runtime support for all pure list, option, and ordered-map intrinsics. - Added dedicated LIR-interpreter and Go-backend test assemblies, and completed higher-order collection intrinsics in LIR execution. - Added compiled local binding patterns, sequential list iteration, and record/union type tokens across LIR artifacts, interpretation, and Go generation. - Split LIR execution and Go generation into independent `FScript.Interpreter.Lir` and `FScript.Backend.Go` assemblies. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 6ae056a..eebe219 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -32,6 +32,8 @@ Language builtins are explicit `Intrinsic` declarations rather than magic unreso The LIR interpreter implements the pure builtin catalog, including callbacks for list mapping, selection, collection, folding, filtering and search; lazy option fallback and mapping; and ordered-map construction, folds, transforms, iteration, and removal. Callback application goes through the same curried closure path as ordinary source calls, preserving failure propagation and evaluation order. +Generated Go represents every FScript function as an erased `fsruntime.Closure` with a single `any -> any` call slot. Typed values are checked at closure entry and result use. This uniform convention permits ordinary source closures, partial application, recursion, and higher-order runtime intrinsics to interoperate without Go generics or reflection-based invocation. + Host externs are copied into IR as name, signature, and arity only; their executable .NET delegates never cross the compiler boundary. Constructing or retaining an unused extern handle is inert. Fully applying one in the LIR interpreter produces `FSI3021`, preserving the rule that unreachable unsupported linkage does not block execution. Local binding patterns reuse the same single-scrutinee decision-tree representation as `match`. Sequential `for ... in ... do ...` is explicit in HIR and LIR so each consumer evaluates the input list once, visits values in order, and discards each body result before returning unit. `typeof` resolves declared record and union definitions during HIR construction and carries the semantic type as a serializable type-token node. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 5dab6fe..dae93d3 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -41,6 +41,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E ## Builtins, externs, attributes, and tasks - Scalar conversions, string helpers, `List`, `Option`, and `Map` functions are protected builtin symbols with explicit polymorphic schemes. They are initial compiler/runtime intrinsics even where a future implementation can compile their FScript source. +- The LIR interpreter and Go runtime execute the complete pure builtin catalog, including higher-order list, option, and ordered-map functions. Callback invocation remains curried and follows source collection order. - `Console.writeLine`, `Console.readLine`, and the CLI `Env` value are candidate portable runtime primitives. Filesystem, regex, hashing, GUID, JSON, and XML functions remain host externs initially. - `Task.spawn` and `Task.await` are host externs with tracked-task semantics, background error propagation, and an error for unawaited tasks. HIR may represent them before LIR interpreters or backends support them. - `[]` is declaration metadata used by embedding hosts and does not change inference. Backend `Import` metadata is separate and will not weaken the declared FScript type. diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index 883aaeb..660509e 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -16,7 +16,7 @@ module GoBackend = | TString -> "string" | TTypeToken -> "string" | TVar _ -> "any" - | TFun(argument, result) -> $"func({goType argument}) {goType result}" + | TFun _ -> "fsruntime.Closure" | TTuple _ -> "[]any" | TList _ -> "[]any" | TOption _ -> "fsruntime.Option" @@ -61,6 +61,7 @@ module GoBackend = | TOption _ -> $"fsruntime.AsOption({code})" | TUnion _ | TNamed _ -> $"fsruntime.AsUnion({code})" | TMap _ -> $"fsruntime.AsMap({code})" + | TFun _ -> $"fsruntime.AsClosure({code})" | _ -> code let rec private patternBindings (pattern: Lir.Pattern) = @@ -115,22 +116,22 @@ module GoBackend = let private emitIntrinsic name intrinsicType = let quoted = System.Text.Json.JsonSerializer.Serialize(name) let rec build index arguments = function - | TFun(argumentType, resultType) -> + | TFun(_, resultType) -> let argument = $"intrinsicArg{index}" let body = build (index + 1) (arguments @ [ argument ]) resultType - $"func({argument} {goType argumentType}) {goType resultType} {{ return {body} }}" + $"fsruntime.Closure{{Call: func({argument} any) any {{ return {body} }}}}" | resultType -> let values = String.concat ", " arguments - cast resultType $"fsruntime.Intrinsic({quoted}, []any{{{values}}})" + cast resultType $"fsruntime.DispatchIntrinsic({quoted}, []any{{{values}}})" build 0 [] intrinsicType let private emitHostExternal name externalType = let quoted = System.Text.Json.JsonSerializer.Serialize(name) let rec build index arguments = function - | TFun(argumentType, resultType) -> + | TFun(_, resultType) -> let argument = $"externalArg{index}" let body = build (index + 1) (arguments @ [ argument ]) resultType - $"func({argument} {goType argumentType}) {goType resultType} {{ return {body} }}" + $"fsruntime.Closure{{Call: func({argument} any) any {{ return {body} }}}}" | resultType -> let values = String.concat ", " arguments cast resultType $"fsruntime.Intrinsic(\"__host:\" + {quoted}, []any{{{values}}})" @@ -145,11 +146,12 @@ module GoBackend = |> Option.defaultValue (Error(sprintf "Unknown symbol %%%d" (Lir.symbolNumber id))) | Lir.MakeClosure(parameter, _, parameterType, body) -> let name = identifier "arg" parameter + let parameterValue = cast parameterType "rawValue" emitExpression (environment |> Map.add parameter name) body - |> Result.map (fun bodyCode -> $"func({name} {goType parameterType}) {goType body.Type} {{ return {bodyCode} }}") + |> Result.map (fun bodyCode -> $"fsruntime.Closure{{Call: func(rawValue any) any {{ {name} := {parameterValue}; _ = {name}; return {bodyCode} }}}}") | Lir.CallClosure(fn, argument) -> match emitExpression environment fn, emitExpression environment argument with - | Ok fnCode, Ok argumentCode -> Ok $"({fnCode})({argumentCode})" + | Ok fnCode, Ok argumentCode -> Ok(cast expression.Type $"fsruntime.Call({fnCode}, {argumentCode})") | Error error, _ | _, Error error -> Error error | Lir.Branch(condition, whenTrue, whenFalse) -> match emitExpression environment condition, emitExpression environment whenTrue, emitExpression environment whenFalse with @@ -276,6 +278,41 @@ module GoBackend = results |> List.fold (fun state result -> match state, result with Ok values, Ok value -> Ok(values @ [ value ]) | Error error, _ | _, Error error -> Error error) (Ok []) + let private closureRuntime = """package fsruntime + +type Closure struct { Call func(any) any } + +func AsClosure(value any) Closure { return value.(Closure) } +func Call(function any, argument any) any { return AsClosure(function).Call(argument) } + +func DispatchIntrinsic(name string, args []any) any { + switch name { + case "List.map": callback, values := AsClosure(args[0]), List(args[1]); result := make([]any, len(values)); for index, value := range values { result[index] = callback.Call(value) }; return result + case "List.iter": callback := AsClosure(args[0]); for _, value := range List(args[1]) { callback.Call(value) }; return struct{}{} + case "List.choose": callback, result := AsClosure(args[0]), []any{}; for _, value := range List(args[1]) { chosen := AsOption(callback.Call(value)); if chosen.HasValue { result = append(result, chosen.Value) } }; return result + case "List.collect": callback, result := AsClosure(args[0]), []any{}; for _, value := range List(args[1]) { result = append(result, List(callback.Call(value))...) }; return result + case "List.exists": callback := AsClosure(args[0]); for _, value := range List(args[1]) { if Bool(callback.Call(value)) { return true } }; return false + case "List.distinct": result := []any{}; for _, value := range List(args[0]) { found := false; for _, existing := range result { if Equal(existing, value) { found = true; break } }; if !found { result = append(result, value) } }; return result + case "List.fold": callback, state := AsClosure(args[0]), args[1]; for _, value := range List(args[2]) { state = Call(callback.Call(state), value) }; return state + case "List.filter": callback, result := AsClosure(args[0]), []any{}; for _, value := range List(args[1]) { if Bool(callback.Call(value)) { result = append(result, value) } }; return result + case "List.tryFind": callback := AsClosure(args[0]); for _, value := range List(args[1]) { if Bool(callback.Call(value)) { return Some(value) } }; return None() + case "List.tryGet": callback := AsClosure(args[0]); for index, value := range List(args[1]) { if Bool(callback.Call(value)) { return Some(int64(index)) } }; return None() + case "List.tryItem": index, values := Int(args[0]), List(args[1]); if index < 0 || index >= int64(len(values)) { return None() }; return Some(values[index]) + case "Option.defaultWith": option := AsOption(args[1]); if option.HasValue { return option.Value }; return Call(args[0], struct{}{}) + case "Option.map": option := AsOption(args[1]); if !option.HasValue { return None() }; return Some(Call(args[0], option.Value)) + case "Map.add": values := append([]MapEntry{}, AsMap(args[2]).Entries...); return OrderedMap{Entries: mapSet(values, args[0], args[1], true)} + case "Map.ofList": values := []MapEntry{}; for _, pair := range List(args[0]) { tuple := List(pair); values = mapSet(values, tuple[0], tuple[1], true) }; return OrderedMap{Entries: values} + case "Map.fold": callback, state := AsClosure(args[0]), args[1]; for _, entry := range AsMap(args[2]).Entries { state = Call(Call(callback.Call(state), entry.Key), entry.Value) }; return state + case "Map.filter": callback, result := AsClosure(args[0]), []MapEntry{}; for _, entry := range AsMap(args[1]).Entries { if Bool(Call(callback.Call(entry.Key), entry.Value)) { result = append(result, entry) } }; return OrderedMap{Entries: result} + case "Map.choose": callback, result := AsClosure(args[0]), []MapEntry{}; for _, entry := range AsMap(args[1]).Entries { chosen := AsOption(Call(callback.Call(entry.Key), entry.Value)); if chosen.HasValue { result = append(result, MapEntry{Key: entry.Key, Value: chosen.Value}) } }; return OrderedMap{Entries: result} + case "Map.map": callback, result := AsClosure(args[0]), []MapEntry{}; for _, entry := range AsMap(args[1]).Entries { result = append(result, MapEntry{Key: entry.Key, Value: callback.Call(entry.Value)}) }; return OrderedMap{Entries: result} + case "Map.iter": callback := AsClosure(args[0]); for _, entry := range AsMap(args[1]).Entries { Call(callback.Call(entry.Key), entry.Value) }; return struct{}{} + case "Map.remove": result := []MapEntry{}; for _, entry := range AsMap(args[1]).Entries { if !Equal(entry.Key, args[0]) { result = append(result, entry) } }; return OrderedMap{Entries: result} + default: return Intrinsic(name, args) + } +} +""" + let generate options (program: Lir.Program) : Result = try let names = @@ -309,6 +346,7 @@ module GoBackend = [ { RelativePath = "go.mod"; Content = $"module {moduleName}\n\ngo 1.25\n" } { RelativePath = "main.go"; Content = $"package main\n\nimport (\n\t\"fmt\"\n\t\"{moduleName}/internal/fscriptprogram\"\n\t\"{moduleName}/internal/fsruntime\"\n)\n\nfunc main() {{ {printExpression} }}\n" } { RelativePath = "internal/fscriptprogram/program.go"; Content = programSource } + { RelativePath = "internal/fsruntime/closures.go"; Content = closureRuntime } { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"sort\"\n\t\"strconv\"\n\t\"strings\"\n)\n\ntype Option struct { HasValue bool; Value any }\ntype Union struct { TypeName string; CaseName string; HasValue bool; Value any }\ntype MapEntry struct { Key any; Value any }\ntype OrderedMap struct { Entries []MapEntry }\ntype MapPart struct { IsSpread bool; Key any; Value any; Map OrderedMap }\n\nfunc Unit(value any) struct{} { return struct{}{} }\nfunc Int(value any) int64 { return value.(int64) }\nfunc Float(value any) float64 { return value.(float64) }\nfunc Bool(value any) bool { return value.(bool) }\nfunc String(value any) string { return value.(string) }\nfunc List(value any) []any { return value.([]any) }\nfunc Record(value any) map[string]any { return value.(map[string]any) }\nfunc AsOption(value any) Option { return value.(Option) }\nfunc AsUnion(value any) Union { return value.(Union) }\nfunc AsMap(value any) OrderedMap { return value.(OrderedMap) }\nfunc Some(value any) Option { return Option{HasValue: true, Value: value} }\nfunc None() Option { return Option{} }\nfunc IsSome(value any) bool { return AsOption(value).HasValue }\nfunc OptionValue(value any) any { return AsOption(value).Value }\nfunc MakeUnion(typeName string, caseName string) Union { return Union{TypeName: typeName, CaseName: caseName} }\nfunc MakeUnionValue(typeName string, caseName string, value any) Union { return Union{TypeName: typeName, CaseName: caseName, HasValue: true, Value: value} }\nfunc IsUnion(value any, typeName string, caseName string) bool { item := AsUnion(value); return item.TypeName == typeName && item.CaseName == caseName }\nfunc UnionHasValue(value any) bool { return AsUnion(value).HasValue }\nfunc UnionValue(value any) any { return AsUnion(value).Value }\nfunc HasField(value any, name string) bool { _, ok := Record(value)[name]; return ok }\nfunc Equal(left any, right any) bool { return reflect.DeepEqual(left, right) }\nfunc Cons(head any, tail []any) []any { return append([]any{head}, tail...) }\nfunc Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) }\nfunc Tail(value any) []any { return append([]any{}, List(value)[1:]...) }\nfunc Range(first int64, last int64) []any { step := int64(1); if first > last { step = -1 }; values := []any{}; for current := first; ; current += step { values = append(values, current); if current == last { break } }; return values }\nfunc UpdateRecord(value any, updates map[string]any) map[string]any { result := map[string]any{}; for key, item := range Record(value) { result[key] = item }; for key, item := range updates { result[key] = item }; return result }\nfunc Pair(key any, value any) MapPart { return MapPart{Key: key, Value: value} }\nfunc SpreadMap(value any) MapPart { return MapPart{IsSpread: true, Map: AsMap(value)} }\nfunc mapSet(entries []MapEntry, key any, value any, replace bool) []MapEntry { for index, entry := range entries { if Equal(entry.Key, key) { if replace { entries[index].Value = value }; return entries } }; return append(entries, MapEntry{Key: key, Value: value}) }\nfunc BuildMap(parts []MapPart) OrderedMap { entries := []MapEntry{}; for _, part := range parts { if part.IsSpread { for _, entry := range part.Map.Entries { entries = mapSet(entries, entry.Key, entry.Value, false) } } else { entries = mapSet(entries, part.Key, part.Value, true) } }; sort.SliceStable(entries, func(left int, right int) bool { return Format(entries[left].Key) < Format(entries[right].Key) }); return OrderedMap{Entries: entries} }\nfunc Index(value any, index any) Option { switch target := value.(type) { case []any: position := Int(index); if position < 0 || position >= int64(len(target)) { return None() }; return Some(target[position]); case OrderedMap: for _, entry := range target.Entries { if Equal(entry.Key, index) { return Some(entry.Value) } }; return None(); default: panic(\"Index access requires a list or map\") } }\nfunc Intrinsic(name string, args []any) any { switch name { case \"ignore\": return struct{}{}; case \"List.empty\": return []any{}; case \"Map.empty\": return OrderedMap{}; case \"Int.tryParse\": value, error := strconv.ParseInt(String(args[0]), 10, 64); if error != nil { return None() }; return Some(value); case \"Float.tryParse\": value, error := strconv.ParseFloat(String(args[0]), 64); if error != nil { return None() }; return Some(value); case \"Bool.tryParse\": value, error := strconv.ParseBool(String(args[0])); if error != nil { return None() }; return Some(value); case \"Int.toString\", \"Float.toString\", \"Bool.toString\": return Format(args[0]); case \"String.replace\": return strings.ReplaceAll(String(args[2]), String(args[0]), String(args[1])); case \"String.indexOf\": index := strings.Index(String(args[1]), String(args[0])); if index < 0 { return None() }; return Some(int64(index)); case \"String.toLower\": return strings.ToLower(String(args[0])); case \"String.toUpper\": return strings.ToUpper(String(args[0])); case \"String.substring\": start, length, value := Int(args[0]), Int(args[1]), String(args[2]); if start < 0 || length < 0 || start + length > int64(len(value)) { return None() }; return Some(value[start:start+length]); case \"String.concat\": values := List(args[1]); parts := make([]string, len(values)); for index, value := range values { parts[index] = String(value) }; return strings.Join(parts, String(args[0])); case \"String.split\": values := strings.Split(String(args[1]), String(args[0])); result := make([]any, len(values)); for index, value := range values { result[index] = value }; return result; case \"String.endsWith\": return strings.HasSuffix(String(args[1]), String(args[0])); case \"List.length\": return int64(len(List(args[0]))); case \"List.rev\": values := List(args[0]); result := make([]any, len(values)); for index, value := range values { result[len(values)-index-1] = value }; return result; case \"List.tryHead\": values := List(args[0]); if len(values) == 0 { return None() }; return Some(values[0]); case \"List.tail\": return Tail(args[0]); case \"List.append\": return Append(List(args[0]), List(args[1])); case \"List.contains\": for _, value := range List(args[1]) { if Equal(args[0], value) { return true } }; return false; case \"Option.isNone\": return !IsSome(args[0]); case \"Option.isSome\": return IsSome(args[0]); case \"Option.defaultValue\": option := AsOption(args[1]); if option.HasValue { return option.Value }; return args[0]; case \"Map.count\": return int64(len(AsMap(args[0]).Entries)); case \"Map.containsKey\": return IsSome(Index(args[1], args[0])); case \"Map.tryGet\": return Index(args[1], args[0]); default: panic(\"Unsupported FScript intrinsic: \" + name) } }\nfunc Format(value any) string { switch item := value.(type) { case bool: if item { return \"true\" }; return \"false\"; case Option: if !item.HasValue { return \"None\" }; return \"Some \" + Format(item.Value); case Union: if !item.HasValue { return item.CaseName }; return item.CaseName + \" \" + Format(item.Value); case OrderedMap: parts := make([]string, len(item.Entries)); for index, entry := range item.Entries { parts[index] = \"[\" + Format(entry.Key) + \"] = \" + Format(entry.Value) }; return \"{ \" + strings.Join(parts, \"; \") + \" }\"; case []any: parts := make([]string, len(item)); for index, value := range item { parts[index] = Format(value) }; return \"[\" + strings.Join(parts, \";\") + \"]\"; default: return fmt.Sprint(value) } }\nfunc Interpolate(values []any) string { var result strings.Builder; for _, value := range values { result.WriteString(Format(value)) }; return result.String() }\nfunc Fail(value any) { panic(fmt.Sprint(value)) }\n" } ] with ex -> Error [ { Code = "FSB4002"; Stage = GoGeneration; Message = ex.Message; Span = None } ] diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index 8ec196e..582a4c6 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -28,6 +28,24 @@ type GoBackendTests() = [] [] [] + [ value + 1) [1; 2]", "[2;3]")>] + [ if value > 1 then Some value else None) [1; 2; 3]", "[2;3]")>] + [ [value; value]) [1; 2]", "[1;1;2;2]")>] + [ state + value) 0 [1; 2; 3]", "6")>] + [ value = 2) [1; 2; 3]", "Some 1")>] + [ value + 1) (Some 4)", "Some 5")>] + [ 7) None", "7")>] + [ state + value) 0 { [\"a\"] = 1; [\"b\"] = 2 }", "3")>] + [ ignore value) [1; 2]", "()")>] + [ value = 2) [1; 2]", "true")>] + [] + [ value > 1) [1; 2; 3]", "[2;3]")>] + [ value > 1) [1; 2; 3]", "Some 2")>] + [] + [] + [ value + 1) (Map.filter (fun key value -> value > 1) (Map.ofList [(\"b\", 2); (\"a\", 1)]))", "{ [b] = 3 }")>] + [ if value > 1 then Some (value + 1) else None) { [\"a\"] = 1; [\"b\"] = 2 }", "{ [b] = 3 }")>] + [] member _.``Generated projects run with Go`` (source: string, expected: string) = let input = { Program = Parser.parseProgram source; Externs = [] } let compiled = From 0a64b7b4c98710cfd00452085e392965200f211c Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 17:34:31 +0200 Subject: [PATCH 021/133] feat(compiler): preserve top-level execution order --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 + docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Compiler/HirLowering.fs | 46 +++++++++++++++++-- .../GoBackendTests.fs | 1 + .../LirInterpreterTests.fs | 16 +++++++ 6 files changed, 64 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a209d35..ba34ae4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Preserved source-order execution for every top-level expression and added compiled top-level pattern bindings. - Added a universal erased Go closure convention and generated-runtime support for all pure list, option, and ordered-map intrinsics. - Added dedicated LIR-interpreter and Go-backend test assemblies, and completed higher-order collection intrinsics in LIR execution. - Added compiled local binding patterns, sequential list iteration, and record/union type tokens across LIR artifacts, interpretation, and Go generation. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index eebe219..1451e4c 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -38,6 +38,8 @@ Host externs are copied into IR as name, signature, and arity only; their execut Local binding patterns reuse the same single-scrutinee decision-tree representation as `match`. Sequential `for ... in ... do ...` is explicit in HIR and LIR so each consumer evaluates the input list once, visits values in order, and discards each body result before returning unit. `typeof` resolves declared record and union definitions during HIR construction and carries the semantic type as a serializable type-token node. +Every top-level source expression becomes a synthetic declaration at its exact position in the declaration stream; the program entry references the final such value. This preserves eager source ordering without evaluating the result expression twice. Top-level pattern bindings similarly store their scrutinee in one synthetic declaration, validate the pattern, and publish deterministic projections for each bound name. + Serialized `.fsir` files contain one closed resolved program plus explicit foreign linkage declarations. JSON format versions are internal and may be rejected rather than upgraded. ## Representation policy diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index dae93d3..b16f2a6 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -10,6 +10,7 @@ This document records the behavior that the typed IR, LIR interpreter, and compi - Hindley-Milner inference generalizes top-level and local let bindings after applying substitutions. Recursive bindings are inferred monomorphically inside their recursive group and generalized afterward. - The active typed program stores top-level types but retains parser expressions in bodies. Compiler lowering therefore requires a new resolved, recursively typed result emitted by the inference pass; types cannot safely be reconstructed afterward. - Top-level statements execute in source order. Every non-final expression must have type `unit`; the final expression is the program result. +- Compiled top-level expressions execute exactly once in source order even when interleaved with declarations. Top-level pattern bindings evaluate their scrutinee once, fail before later statements when unmatched, and expose all bound names to subsequent statements. The first aggregate compiler slice preserves tuple field order, record field names, option presence, list element order, list cons/append behavior, and recursively structural equality through HIR, serialized LIR, interpretation, and Go generation. diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index 4047c11..4d1d422 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -316,6 +316,7 @@ module HirLowering = resolved.TypedProgram |> List.collect (function | TypeInfer.TSLet(name, _, _, _, _, _) -> [ name ] + | TypeInfer.TSLetPattern(_, _, bindings, _, _) -> bindings |> Map.keys |> Seq.toList | TypeInfer.TSLetRecGroup(bindings, _, _) -> bindings |> List.map (fun (name, _, _, _) -> name) | _ -> []) |> List.distinct @@ -345,7 +346,7 @@ module HirLowering = let body = ({ Type = unionType; Span = definition.Span; Node = Hir.UnionCase(typeName, caseName, Some({ Type = itemType; Span = definition.Span; Node = Hir.Reference parameter })) }: Hir.Expression) ({ Type = TFun(itemType, unionType); Span = definition.Span; Node = Hir.Lambda(parameter, "value", itemType, body) }: Hir.Expression) ({ Id = id; Name = $"{typeName}.{caseName}"; Scheme = Types.generalize Map.empty value.Type; IsRecursive = false; IsExported = false; Span = definition.Span; Value = value }: Hir.Declaration))) - let mutable entry = None + let mutable entry: Hir.Expression option = None let mutable errors = [] for statement in resolved.TypedProgram do match statement with @@ -363,8 +364,48 @@ module HirLowering = | Error diagnostics -> errors <- errors @ diagnostics | TypeInfer.TSExpr expression -> match lowerExpression state topLevel expression.Type expression.Expr with - | Ok value -> entry <- Some value + | Ok value -> + let id = newSymbol state + let name = $"$expression{Hir.symbolNumber id}" + declarations <- declarations @ + [ ({ Id = id + Name = name + Scheme = Types.generalize Map.empty expression.Type + IsRecursive = false + IsExported = false + Span = value.Span + Value = value }: Hir.Declaration) ] + entry <- Some { Type = expression.Type; Span = value.Span; Node = Hir.Reference id } | Error diagnostics -> errors <- errors @ diagnostics + | TypeInfer.TSLetPattern(pattern, expression, bindings, isExported, span) -> + let bindingType = expressionType state expression (Types.freshVar()) + match lowerExpression state topLevel bindingType expression, lowerPattern state bindingType pattern with + | Ok value, Ok(loweredPattern, patternBindings) -> + let temporaryId = newSymbol state + declarations <- declarations @ + [ ({ Id = temporaryId + Name = $"$pattern{Hir.symbolNumber temporaryId}" + Scheme = Types.generalize Map.empty bindingType + IsRecursive = false + IsExported = false + Span = span + Value = value }: Hir.Declaration) ] + let target = ({ Type = bindingType; Span = span; Node = Hir.Reference temporaryId }: Hir.Expression) + let unitBody = ({ Type = TUnit; Span = span; Node = Hir.Constant Hir.Unit }: Hir.Expression) + let check = ({ Type = TUnit; Span = span; Node = Hir.Match(target, [ { Pattern = loweredPattern; Guard = None; Body = unitBody; Span = span } ]) }: Hir.Expression) + let checkId = newSymbol state + declarations <- declarations @ + [ ({ Id = checkId; Name = $"$patternCheck{Hir.symbolNumber checkId}"; Scheme = Forall([], TUnit); IsRecursive = false; IsExported = false; Span = span; Value = check }: Hir.Declaration) ] + for name, inferred in bindings |> Map.toList do + match patternBindings |> Map.tryFind name with + | None -> errors <- errors @ [ Diagnostic.hir "FSH1006" $"Pattern binding '{name}' was not resolved." (Some span) ] + | Some patternId -> + let body = ({ Type = inferred; Span = span; Node = Hir.Reference patternId }: Hir.Expression) + let projection = ({ Type = inferred; Span = span; Node = Hir.Match(target, [ { Pattern = loweredPattern; Guard = None; Body = body; Span = span } ]) }: Hir.Expression) + declarations <- declarations @ + [ ({ Id = Map.find name topLevel; Name = name; Scheme = Types.generalize Map.empty inferred; IsRecursive = false; IsExported = isExported; Span = span; Value = projection }: Hir.Declaration) ] + | Error a, Error b -> errors <- errors @ a @ b + | Error diagnostics, _ | _, Error diagnostics -> errors <- errors @ diagnostics | TypeInfer.TSLetRecGroup(bindings, isExported, groupSpan) -> for name, expression, inferred, span in bindings do match lowerExpression state topLevel inferred expression with @@ -379,6 +420,5 @@ module HirLowering = Value = value }: Hir.Declaration) ] | Error diagnostics -> errors <- errors @ diagnostics | TypeInfer.TSType _ -> () - | _ -> errors <- errors @ [ Diagnostic.hir "FSH1098" "Top-level declaration form is not yet supported by HIR lowering." None ] if errors.IsEmpty then Ok { Declarations = declarations; Entry = entry } else Error errors diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index 582a4c6..2fcec6f 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -46,6 +46,7 @@ type GoBackendTests() = [ value + 1) (Map.filter (fun key value -> value > 1) (Map.ofList [(\"b\", 2); (\"a\", 1)]))", "{ [b] = 3 }")>] [ if value > 1 then Some (value + 1) else None) { [\"a\"] = 1; [\"b\"] = 2 }", "{ [b] = 3 }")>] [] + [] member _.``Generated projects run with Go`` (source: string, expected: string) = let input = { Program = Parser.parseProgram source; Externs = [] } let compiled = diff --git a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs index a52fb0f..23140fc 100644 --- a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs +++ b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs @@ -172,3 +172,19 @@ type LirInterpreterTests() = match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(expected), source) | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``Preserves top-level pattern bindings and expression order`` () = + match Compiler.compile { Program = Parser.parseProgram "let (left, right) = (1, 2)\nleft + right"; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 3L)) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + match Compiler.compile { Program = Parser.parseProgram "raise \"first\"\nraise \"second\""; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok _ -> Assert.Fail("Expected the first top-level expression to fail") + | Error diagnostics -> Assert.That(diagnostics.Head.Message, Is.EqualTo("first")) From 19f98102c2d2611454cc457e03b6635b9cf97cde Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 17:38:12 +0200 Subject: [PATCH 022/133] feat(compiler): execute local recursive groups --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 ++ docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Backend.Go/GoBackend.fs | 12 ++++++++++++ src/FScript.Compiler/Hir.fs | 5 +++++ src/FScript.Compiler/HirLowering.fs | 16 +++++++++++++++- src/FScript.Compiler/HirValidation.fs | 6 ++++++ src/FScript.Compiler/IrArtifact.fs | 12 ++++++++++++ src/FScript.Compiler/Lir.fs | 3 +++ src/FScript.Interpreter.Lir/LirInterpreter.fs | 10 ++++++++++ tests/FScript.Backend.Go.Tests/GoBackendTests.fs | 1 + tests/FScript.Compiler.Tests/IrArtifactTests.fs | 3 ++- .../LirInterpreterTests.fs | 10 ++++++++++ 13 files changed, 80 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ba34ae4..59b198f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added serializable local mutual-recursive closure groups to HIR, LIR execution, and generated Go projects. - Preserved source-order execution for every top-level expression and added compiled top-level pattern bindings. - Added a universal erased Go closure convention and generated-runtime support for all pure list, option, and ordered-map intrinsics. - Added dedicated LIR-interpreter and Go-backend test assemblies, and completed higher-order collection intrinsics in LIR execution. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 1451e4c..bc77964 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -28,6 +28,8 @@ Matches lower to a structured decision tree whose target is evaluated once. Case Top-level mutual-recursive declarations are initialized as an atomic closure group. The interpreter creates every closure against the preceding environment, extends that environment with the complete group, and updates all captures together. Generated Go declares typed slots for all declarations and assigns them in source order from `init`, allowing closures in one recursive group to refer to every peer. +Local `let rec ... and ...` uses the same atomic rule inside a structured expression node. All local names are allocated before any member is lowered, every binding must evaluate to a closure, captures are updated with the complete group, and the body runs only after group initialization. + Language builtins are explicit `Intrinsic` declarations rather than magic unresolved references. Consumers accumulate their curried arguments according to the catalog signature and dispatch only at full application. The initial portable implementations cover scalar parsing/formatting, strings, simple list and option operations, and map queries; a reachable catalog entry without an implementation receives a stage-specific unsupported diagnostic or runtime failure. The LIR interpreter implements the pure builtin catalog, including callbacks for list mapping, selection, collection, folding, filtering and search; lazy option fallback and mapping; and ordered-map construction, folds, transforms, iteration, and removal. Callback application goes through the same curried closure path as ordinary source calls, preserving failure propagation and evaluation order. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index b16f2a6..94a7480 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -20,6 +20,7 @@ Declared unions retain their type and case identity plus an optional erased payl Compiled maps use an ordered entry representation for string and integer keys. Explicit key/value entries replace an earlier value; spread entries retain values already present on their left. Indexing returns an option, and deterministic ordering participates in artifacts, equality, formatting, and generated Go behavior. - Functions are curried. Multi-argument declarations are nested lambdas, application is one argument at a time, and closures capture their lexical environment. Single and mutual recursive function groups are supported. +- Local mutual-recursive function groups initialize atomically: every peer is visible from every closure before the group body executes. ## Types and values diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index 660509e..9d817be 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -203,6 +203,18 @@ module GoBackend = else $"{name} := {bindingCode}" Ok $"func() {goType expression.Type} {{ {declaration}; return {bodyCode} }}()" | Error error, _ | _, Error error -> Error error + | Lir.LetRecursiveGroup(bindings, body) -> + let scoped = bindings |> List.fold (fun env (id, _, _, _) -> Map.add id (identifier "localRec" id) env) environment + let loweredBindings = + bindings |> List.map (fun (id, _, bindingType, value) -> + emitExpression scoped value |> Result.map (fun code -> identifier "localRec" id, bindingType, code)) + |> collect + match loweredBindings, emitExpression scoped body with + | Ok values, Ok bodyCode -> + let declarations = values |> List.map (fun (name, bindingType, _) -> $"var {name} {goType bindingType}") |> String.concat "; " + let assignments = values |> List.map (fun (name, _, code) -> $"{name} = {code}") |> String.concat "; " + Ok $"func() {goType expression.Type} {{ {declarations}; {assignments}; return {bodyCode} }}()" + | Error error, _ | _, Error error -> Error error | Lir.Fail value -> emitExpression environment value |> Result.map (fun code -> $"func() {goType expression.Type} {{ fsruntime.Fail({code}); var zero {goType expression.Type}; return zero }}()") diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs index 5de08e2..30395f1 100644 --- a/src/FScript.Compiler/Hir.fs +++ b/src/FScript.Compiler/Hir.fs @@ -51,6 +51,7 @@ module Hir = | Cons of Expression * Expression | Append of Expression * Expression | Let of SymbolId * string * bool * Expression * Expression + | LetRecursiveGroup of (SymbolId * string * Type * Expression) list * Expression | Fail of Expression | Match of Expression * MatchCase list | Range of Expression * Expression @@ -160,6 +161,10 @@ module Hir = lines.Add(typed (sprintf "let%s %%%d %s" (if recursive then " rec" else "") (symbolNumber id) name)) expression (indent + 2) binding expression (indent + 2) body + | LetRecursiveGroup(bindings, body) -> + lines.Add(typed "let-rec-group") + bindings |> List.iter (fun (id, name, _, binding) -> lines.Add(sprintf "%s binding %%%d %s" prefix (symbolNumber id) name); expression (indent + 4) binding) + expression (indent + 2) body | Fail value -> lines.Add(typed "fail"); expression (indent + 2) value | Match(target, cases) -> lines.Add(typed $"match {cases.Length} cases") diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index 4d1d422..6f4c82b 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -205,6 +205,21 @@ module HirLowering = | Ok value, Ok loweredBody -> Ok(make (Hir.Let(id, name, isRecursive, value, loweredBody))) | Error a, Error b -> Error(a @ b) | Error errors, _ | _, Error errors -> Error errors + | ELetRecGroup(bindings, body, _) -> + let ids = bindings |> List.map (fun (name, _, _, _, _) -> name, newSymbol state) |> Map.ofList + let recursiveEnvironment = Map.fold (fun env name id -> Map.add name id env) environment ids + let loweredBindings = + bindings + |> List.map (fun (name, parameters, _, valueExpression, bindingSpan) -> + let folded = Seq.foldBack (fun parameter inner -> ELambda(parameter, inner, bindingSpan)) parameters valueExpression + let bindingType = expressionType state folded (Types.freshVar()) + lowerExpression state recursiveEnvironment bindingType folded + |> Result.map (fun value -> ids.[name], name, bindingType, value)) + |> collectResults + match loweredBindings, lowerExpression state recursiveEnvironment inferred body with + | Ok values, Ok loweredBody -> Ok(make (Hir.LetRecursiveGroup(values, loweredBody))) + | Error a, Error b -> Error(a @ b) + | Error errors, _ | _, Error errors -> Error errors | ERaise(value, _) -> lowerExpression state environment TString value |> Result.map (Hir.Fail >> make) | ELetPattern(pattern, binding, body, _) -> let bindingType = expressionType state binding (Types.freshVar()) @@ -269,7 +284,6 @@ module HirLowering = | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e | MESpread value -> lowerExpression state environment inferred value |> Result.map Hir.Spread) |> collectResults |> Result.map (Hir.Map >> make) - | _ -> Error [ Diagnostic.hir "FSH1099" $"HIR lowering does not yet support {expression.GetType().Name}." (Some span) ] let lower (externs: ExternalFunction list) (resolved: TypeInfer.ResolvedProgram) : Result = let expressionTypes = diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index c4fe91d..ef725da 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -26,6 +26,12 @@ module HirValidation = let value = references binding if recursive then Set.remove id value else value Set.union bindingReferences (references body |> Set.remove id) + | Hir.LetRecursiveGroup(bindings, body) -> + let ids = bindings |> List.map (fun (id, _, _, _) -> id) |> Set.ofList + let allReferences = + references body :: (bindings |> List.map (fun (_, _, _, value) -> references value)) + |> Set.unionMany + Set.difference allReferences ids | Hir.Fail value -> references value | Hir.Match(target, cases) -> let rec bindings (pattern: Hir.Pattern) = diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index 7a0af4c..82d876e 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -141,6 +141,13 @@ module IrArtifact = writer.WriteString("kind", "let"); writer.WriteNumber("symbol", Lir.symbolNumber id); writer.WriteString("name", name) writer.WriteBoolean("recursive", recursive); writer.WritePropertyName("binding"); writeExpression writer binding writer.WritePropertyName("body"); writeExpression writer body + | Lir.LetRecursiveGroup(bindings, body) -> + writer.WriteString("kind", "let-rec-group"); writer.WriteStartArray("bindings") + for id, name, bindingType, value in bindings do + writer.WriteStartObject(); writer.WriteNumber("symbol", Lir.symbolNumber id); writer.WriteString("name", name) + writer.WritePropertyName("bindingType"); writeType writer bindingType + writer.WritePropertyName("value"); writeExpression writer value; writer.WriteEndObject() + writer.WriteEndArray(); writer.WritePropertyName("body"); writeExpression writer body | Lir.Fail value -> writer.WriteString("kind", "fail"); writer.WritePropertyName("value"); writeExpression writer value | Lir.DecisionTree(target, cases) -> writer.WriteString("kind", "decision-tree"); writer.WritePropertyName("target"); writeExpression writer target @@ -293,6 +300,11 @@ module IrArtifact = | "append" -> Lir.Append(readExpression (property "left" element), readExpression (property "right" element)) | "let" -> Lir.Let(Lir.SymbolId((property "symbol" element).GetInt32()), stringProperty "name" element, (property "recursive" element).GetBoolean(), readExpression (property "binding" element), readExpression (property "body" element)) + | "let-rec-group" -> + let bindings = (property "bindings" element).EnumerateArray() |> Seq.map (fun item -> + Lir.SymbolId((property "symbol" item).GetInt32()), stringProperty "name" item, + readType (property "bindingType" item), readExpression (property "value" item)) |> Seq.toList + Lir.LetRecursiveGroup(bindings, readExpression (property "body" element)) | "fail" -> Lir.Fail(readExpression (property "value" element)) | "decision-tree" -> let cases = (property "cases" element).EnumerateArray() |> Seq.map (fun item -> diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index 6cfa15f..35ebc96 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -39,6 +39,7 @@ module Lir = | Cons of Expression * Expression | Append of Expression * Expression | Let of SymbolId * string * bool * Expression * Expression + | LetRecursiveGroup of (SymbolId * string * Type * Expression) list * Expression | Fail of Expression | DecisionTree of Expression * DecisionCase list | Range of Expression * Expression @@ -149,6 +150,8 @@ module LirLowering = | Hir.Cons(head, tail) -> Lir.Cons(expression head, expression tail) | Hir.Append(left, right) -> Lir.Append(expression left, expression right) | Hir.Let(id, name, recursive, binding, body) -> Lir.Let(symbol id, name, recursive, expression binding, expression body) + | Hir.LetRecursiveGroup(bindings, body) -> + Lir.LetRecursiveGroup(bindings |> List.map (fun (id, name, bindingType, value) -> symbol id, name, bindingType, expression value), expression body) | Hir.Fail value -> Lir.Fail(expression value) | Hir.Match(target, cases) -> Lir.DecisionTree(expression target, cases |> List.map (fun case -> diff --git a/src/FScript.Interpreter.Lir/LirInterpreter.fs b/src/FScript.Interpreter.Lir/LirInterpreter.fs index 50bd019..c8d48a1 100644 --- a/src/FScript.Interpreter.Lir/LirInterpreter.fs +++ b/src/FScript.Interpreter.Lir/LirInterpreter.fs @@ -274,6 +274,16 @@ module LirInterpreter = | Error errors -> Error errors else evaluate environment binding |> Result.bind (fun value -> evaluate (environment |> Map.add id value) body) + | Lir.LetRecursiveGroup(bindings, body) -> + let evaluated = bindings |> List.map (fun (id, _, _, binding) -> id, evaluate environment binding) + let errors = evaluated |> List.collect (fun (_, result) -> match result with Error values -> values | Ok _ -> []) + if not errors.IsEmpty then Error errors + elif evaluated |> List.exists (fun (_, result) -> match result with Ok(VClosure _) -> false | _ -> true) then + fail "FSI3010" expression.Span "Recursive local groups must contain only functions." + else + let recursiveEnvironment = evaluated |> List.fold (fun env (id, result) -> match result with Ok value -> Map.add id value env | Error _ -> env) environment + evaluated |> List.iter (fun (_, result) -> match result with Ok(VClosure(_, _, _, captured)) -> captured.Value <- recursiveEnvironment | _ -> ()) + evaluate recursiveEnvironment body | Lir.Fail value -> match evaluate environment value with | Ok(VString message) -> fail "FSI3011" expression.Span message diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index 2fcec6f..7e5f6c0 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -47,6 +47,7 @@ type GoBackendTests() = [ if value > 1 then Some (value + 1) else None) { [\"a\"] = 1; [\"b\"] = 2 }", "{ [b] = 3 }")>] [] [] + [] member _.``Generated projects run with Go`` (source: string, expected: string) = let input = { Program = Parser.parseProgram source; Externs = [] } let compiled = diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index b7d34c4..730ac8f 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -49,7 +49,8 @@ type IrArtifactTests() = "{ [\"answer\"] = 42 }[\"answer\"]" "let sum pair = (let (left, right) = pair\n left + right\n)\nsum (1, 2)" "for value in [1; 2; 3] do ignore value" - "type Package = { Name: string }\ntypeof Package" ] do + "type Package = { Name: string }\ntypeof Package" + "(let rec even n = if n = 0 then true else odd (n - 1)\nand odd n = if n = 0 then false else even (n - 1)\neven 10\n)" ] do let bytes = compile source |> _.Lir |> IrArtifact.serialize match IrArtifact.deserialize bytes with | Ok _ -> () diff --git a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs index 23140fc..5c18fe8 100644 --- a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs +++ b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs @@ -188,3 +188,13 @@ type LirInterpreterTests() = match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok _ -> Assert.Fail("Expected the first top-level expression to fail") | Error diagnostics -> Assert.That(diagnostics.Head.Message, Is.EqualTo("first")) + + [] + member _.``Executes local mutual recursive closure groups`` () = + let source = "(let rec even n = if n = 0 then true else odd (n - 1)\nand odd n = if n = 0 then false else even (n - 1)\neven 10\n)" + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VBool true)) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) From a539ad1bff89f2e10f7ee55f8f8efd5d7cf6868a Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 17:43:08 +0200 Subject: [PATCH 023/133] feat(compiler): execute map and type patterns --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 + docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Backend.Go/GoBackend.fs | 91 +++++++++++++++++++ src/FScript.Compiler/Hir.fs | 2 + src/FScript.Compiler/HirLowering.fs | 20 +++- src/FScript.Compiler/HirValidation.fs | 4 + src/FScript.Compiler/IrArtifact.fs | 13 +++ src/FScript.Compiler/Lir.fs | 4 + src/FScript.Interpreter.Lir/LirInterpreter.fs | 37 +++++++- .../GoBackendTests.fs | 2 + .../FScript.Compiler.Tests/IrArtifactTests.fs | 4 +- .../LirInterpreterTests.fs | 10 ++ 13 files changed, 188 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 59b198f..cf452d7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added deterministic ordered-map and runtime type-reference patterns to HIR, artifacts, LIR execution, and generated Go. - Added serializable local mutual-recursive closure groups to HIR, LIR execution, and generated Go projects. - Preserved source-order execution for every top-level expression and added compiled top-level pattern bindings. - Added a universal erased Go closure convention and generated-runtime support for all pure list, option, and ordered-map intrinsics. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index bc77964..14cd914 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -26,6 +26,8 @@ Lexical `let` is retained as a structured LIR node. Its symbol is visible only i Matches lower to a structured decision tree whose target is evaluated once. Cases remain ordered; each pattern test creates artifact-local bindings, then its guard is evaluated, and only then is its body selected. A failed guard continues at the next case. Exhaustion is the explicit FScript match-failure path. Literal, list, tuple, record, and option patterns use the same tree in serialized artifacts, the LIR interpreter, and generated Go. +Ordered-map patterns retain ordered clause matching, repeated-binding consistency, remainder binding, and the existing rule that explicit clauses may leave unmatched entries. Type-reference patterns carry their resolved semantic type; consumers test primitive and aggregate representation shape and retain declared union identity. + Top-level mutual-recursive declarations are initialized as an atomic closure group. The interpreter creates every closure against the preceding environment, extends that environment with the complete group, and updates all captures together. Generated Go declares typed slots for all declarations and assigns them in source order from `init`, allowing closures in one recursive group to refer to every peer. Local `let rec ... and ...` uses the same atomic rule inside a structured expression node. All local names are allocated before any member is lowered, every binding must evaluate to a closure, captures are updated with the complete group, and the body runs only after group initialization. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 94a7480..f038d45 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -37,6 +37,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Conditionals and match guards require Boolean values and evaluate only the selected branch. - Match clauses are tried in source order. A guard runs only after its pattern binds successfully. Failure to find a clause is a runtime error. - Patterns cover literals, variables, wildcards, lists, tuples, records, ordered maps, options, union cases, and type-reference tests. Let-pattern failure is a runtime error. +- Compiled map patterns consume entries deterministically in map order, merge repeated bindings only when structurally equal, and bind an optional tail to the remaining ordered map. Compiled type-reference patterns use resolved types rather than source text lookup. - Record updates evaluate the base once and then updates in source order. Ranges, loops, interpolation, indexing, raise, append, and cons preserve the evaluator's existing evaluation order and failures. - Equality is structural for primitives, tuples, records, maps, options, lists, unions, and type tokens, recursively. Functions, tasks, constructors, and extern handles are not structurally equal. Backend host-language comparability must not leak into this behavior. diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index 9d817be..5fdd1db 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -70,10 +70,51 @@ module GoBackend = | Lir.ConsPattern(head, tail) -> patternBindings head @ patternBindings tail | Lir.TuplePattern items -> List.collect patternBindings items | Lir.RecordPattern fields -> fields |> List.collect (snd >> patternBindings) + | Lir.MapPattern(clauses, tail) -> + clauses |> List.collect (fun (key, value) -> patternBindings key @ patternBindings value) + |> (@) (tail |> Option.map patternBindings |> Option.defaultValue []) | Lir.SomePattern item -> patternBindings item | Lir.UnionPattern(_, _, payload) -> payload |> Option.map patternBindings |> Option.defaultValue [] | _ -> [] + let rec private patternBindingDetails (pattern: Lir.Pattern) = + match pattern.Node with + | Lir.Binding(id, _) -> [ id, pattern.Type ] + | Lir.ConsPattern(head, tail) -> patternBindingDetails head @ patternBindingDetails tail + | Lir.TuplePattern items -> List.collect patternBindingDetails items + | Lir.RecordPattern fields -> fields |> List.collect (snd >> patternBindingDetails) + | Lir.MapPattern(clauses, tail) -> + clauses |> List.collect (fun (key, value) -> patternBindingDetails key @ patternBindingDetails value) + |> (@) (tail |> Option.map patternBindingDetails |> Option.defaultValue []) + | Lir.SomePattern item -> patternBindingDetails item + | Lir.UnionPattern(_, _, payload) -> payload |> Option.map patternBindingDetails |> Option.defaultValue [] + | _ -> [] + + let rec private emitPatternDescriptor (pattern: Lir.Pattern) = + let quoted value = System.Text.Json.JsonSerializer.Serialize(value) + match pattern.Node with + | Lir.Wildcard -> "fsruntime.Pattern{Kind: \"wildcard\"}" + | Lir.Binding(id, _) -> $"fsruntime.Pattern{{Kind: \"binding\", Symbol: {Lir.symbolNumber id}}}" + | Lir.LiteralPattern value -> $"fsruntime.Pattern{{Kind: \"literal\", Literal: {constant value}}}" + | Lir.EmptyList -> "fsruntime.Pattern{Kind: \"empty-list\"}" + | Lir.NonePattern -> "fsruntime.Pattern{Kind: \"none\"}" + | Lir.SomePattern item -> $"fsruntime.Pattern{{Kind: \"some\", Items: []fsruntime.Pattern{{{emitPatternDescriptor item}}}}}" + | Lir.ConsPattern(head, tail) -> $"fsruntime.Pattern{{Kind: \"cons\", Items: []fsruntime.Pattern{{{emitPatternDescriptor head}, {emitPatternDescriptor tail}}}}}" + | Lir.TuplePattern items -> + let values = items |> List.map emitPatternDescriptor |> String.concat ", " + $"fsruntime.Pattern{{Kind: \"tuple\", Items: []fsruntime.Pattern{{{values}}}}}" + | Lir.RecordPattern fields -> + let values = fields |> List.map (fun (name, item) -> $"fsruntime.FieldPattern{{Name: {quoted name}, Pattern: {emitPatternDescriptor item}}}") |> String.concat ", " + $"fsruntime.Pattern{{Kind: \"record\", Fields: []fsruntime.FieldPattern{{{values}}}}}" + | Lir.MapPattern(clauses, tail) -> + let values = clauses |> List.map (fun (key, value) -> $"fsruntime.MapPatternClause{{Key: {emitPatternDescriptor key}, Value: {emitPatternDescriptor value}}}") |> String.concat ", " + let tailValue = tail |> Option.map (fun item -> $"fsruntime.PatternPtr({emitPatternDescriptor item})") |> Option.defaultValue "nil" + $"fsruntime.Pattern{{Kind: \"map\", Clauses: []fsruntime.MapPatternClause{{{values}}}, Tail: {tailValue}}}" + | Lir.UnionPattern(typeName, caseName, payload) -> + let payloadValue = payload |> Option.map (fun item -> $"[]fsruntime.Pattern{{{emitPatternDescriptor item}}}") |> Option.defaultValue "nil" + $"fsruntime.Pattern{{Kind: \"union\", TypeName: {quoted typeName}, CaseName: {quoted caseName}, Items: {payloadValue}}}" + | Lir.TypePattern patternType -> $"fsruntime.Pattern{{Kind: \"type\", TypeName: {quoted (Types.typeToString patternType)}}}" + let rec private emitPattern valueCode (pattern: Lir.Pattern) = let combine parts = let tests, assignments = List.unzip parts @@ -104,6 +145,14 @@ module GoBackend = let test, assignments = emitPattern $"fsruntime.Record({valueCode})[{quoted}]" item $"fsruntime.HasField({valueCode}, {quoted}) && ({test})", assignments) |> combine + | Lir.MapPattern _ -> + let descriptor = emitPatternDescriptor pattern + let assignments = + patternBindingDetails pattern |> List.distinctBy fst |> List.map (fun (id, bindingType) -> + let name = identifier "match" id + let bindingValue = $"fsruntime.PatternBinding({valueCode}, {descriptor}, {Lir.symbolNumber id})" + $"{name} := {cast bindingType bindingValue}; _ = {name}") + $"fsruntime.PatternMatches({valueCode}, {descriptor})", assignments | Lir.UnionPattern(typeName, caseName, payload) -> let quotedType = System.Text.Json.JsonSerializer.Serialize(typeName) let quotedCase = System.Text.Json.JsonSerializer.Serialize(caseName) @@ -112,6 +161,7 @@ module GoBackend = | Some item -> let test, assignments = emitPattern $"fsruntime.UnionValue({valueCode})" item $"fsruntime.IsUnion({valueCode}, {quotedType}, {quotedCase}) && fsruntime.UnionHasValue({valueCode}) && ({test})", assignments + | Lir.TypePattern patternType -> $"fsruntime.TypeMatches({valueCode}, {System.Text.Json.JsonSerializer.Serialize(Types.typeToString patternType)})", [] let private emitIntrinsic name intrinsicType = let quoted = System.Text.Json.JsonSerializer.Serialize(name) @@ -323,6 +373,46 @@ func DispatchIntrinsic(name string, args []any) any { default: return Intrinsic(name, args) } } +""" + + let private patternRuntime = """package fsruntime + +import "strings" + +type Pattern struct { Kind string; Symbol int; Literal any; Items []Pattern; Fields []FieldPattern; Clauses []MapPatternClause; Tail *Pattern; TypeName string; CaseName string } +type FieldPattern struct { Name string; Pattern Pattern } +type MapPatternClause struct { Key Pattern; Value Pattern } + +func PatternPtr(value Pattern) *Pattern { return &value } + +func mergePatternBindings(left map[int]any, right map[int]any) (map[int]any, bool) { + result := map[int]any{}; for key, value := range left { result[key] = value }; for key, value := range right { if existing, found := result[key]; found && !Equal(existing, value) { return nil, false }; result[key] = value }; return result, true +} + +func matchRuntimePattern(value any, pattern Pattern) (map[int]any, bool) { + switch pattern.Kind { + case "wildcard": return map[int]any{}, true + case "binding": return map[int]any{pattern.Symbol: value}, true + case "literal": return map[int]any{}, Equal(value, pattern.Literal) + case "empty-list": values, ok := value.([]any); return map[int]any{}, ok && len(values) == 0 + case "none": option, ok := value.(Option); return map[int]any{}, ok && !option.HasValue + case "some": option, ok := value.(Option); if !ok || !option.HasValue { return nil, false }; return matchRuntimePattern(option.Value, pattern.Items[0]) + case "cons": values, ok := value.([]any); if !ok || len(values) == 0 { return nil, false }; left, matched := matchRuntimePattern(values[0], pattern.Items[0]); if !matched { return nil, false }; right, matched := matchRuntimePattern(values[1:], pattern.Items[1]); if !matched { return nil, false }; return mergePatternBindings(left, right) + case "tuple": values, ok := value.([]any); if !ok || len(values) != len(pattern.Items) { return nil, false }; bindings := map[int]any{}; for index, item := range pattern.Items { next, matched := matchRuntimePattern(values[index], item); if !matched { return nil, false }; bindings, matched = mergePatternBindings(bindings, next); if !matched { return nil, false } }; return bindings, true + case "record": values, ok := value.(map[string]any); if !ok { return nil, false }; bindings := map[int]any{}; for _, field := range pattern.Fields { item, found := values[field.Name]; if !found { return nil, false }; next, matched := matchRuntimePattern(item, field.Pattern); if !matched { return nil, false }; bindings, matched = mergePatternBindings(bindings, next); if !matched { return nil, false } }; return bindings, true + case "union": item, ok := value.(Union); if !ok || item.TypeName != pattern.TypeName || item.CaseName != pattern.CaseName || (len(pattern.Items) == 0) != !item.HasValue { return nil, false }; if len(pattern.Items) == 0 { return map[int]any{}, true }; return matchRuntimePattern(item.Value, pattern.Items[0]) + case "type": return map[int]any{}, TypeMatches(value, pattern.TypeName) + case "map": + item, ok := value.(OrderedMap); if !ok { return nil, false }; remaining := append([]MapEntry{}, item.Entries...); bindings := map[int]any{} + for _, clause := range pattern.Clauses { found := false; for index, entry := range remaining { keyBindings, keyMatched := matchRuntimePattern(entry.Key, clause.Key); if !keyMatched { continue }; withKey, merged := mergePatternBindings(bindings, keyBindings); if !merged { continue }; valueBindings, valueMatched := matchRuntimePattern(entry.Value, clause.Value); if !valueMatched { continue }; withValue, merged := mergePatternBindings(withKey, valueBindings); if !merged { continue }; bindings = withValue; remaining = append(remaining[:index], remaining[index+1:]...); found = true; break }; if !found { return nil, false } } + if pattern.Tail != nil { tailBindings, matched := matchRuntimePattern(OrderedMap{Entries: remaining}, *pattern.Tail); if !matched { return nil, false }; return mergePatternBindings(bindings, tailBindings) }; if len(pattern.Clauses) > 0 || len(remaining) == 0 { return bindings, true }; return nil, false + default: return nil, false + } +} + +func PatternMatches(value any, pattern Pattern) bool { _, matched := matchRuntimePattern(value, pattern); return matched } +func PatternBinding(value any, pattern Pattern, symbol int) any { bindings, matched := matchRuntimePattern(value, pattern); if !matched { panic("Pattern did not match") }; return bindings[symbol] } +func TypeMatches(value any, typeName string) bool { switch typeName { case "unit": _, ok := value.(struct{}); return ok; case "int": _, ok := value.(int64); return ok; case "float": _, ok := value.(float64); return ok; case "bool": _, ok := value.(bool); return ok; case "string": _, ok := value.(string); return ok }; if strings.HasSuffix(typeName, " list") { _, ok := value.([]any); return ok }; if strings.HasSuffix(typeName, " option") { _, ok := value.(Option); return ok }; if strings.HasPrefix(typeName, "{") { _, ok := value.(map[string]any); return ok }; if item, ok := value.(Union); ok { return item.TypeName == typeName }; _, record := value.(map[string]any); return record } """ let generate options (program: Lir.Program) : Result = @@ -359,6 +449,7 @@ func DispatchIntrinsic(name string, args []any) any { { RelativePath = "main.go"; Content = $"package main\n\nimport (\n\t\"fmt\"\n\t\"{moduleName}/internal/fscriptprogram\"\n\t\"{moduleName}/internal/fsruntime\"\n)\n\nfunc main() {{ {printExpression} }}\n" } { RelativePath = "internal/fscriptprogram/program.go"; Content = programSource } { RelativePath = "internal/fsruntime/closures.go"; Content = closureRuntime } + { RelativePath = "internal/fsruntime/patterns.go"; Content = patternRuntime } { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"sort\"\n\t\"strconv\"\n\t\"strings\"\n)\n\ntype Option struct { HasValue bool; Value any }\ntype Union struct { TypeName string; CaseName string; HasValue bool; Value any }\ntype MapEntry struct { Key any; Value any }\ntype OrderedMap struct { Entries []MapEntry }\ntype MapPart struct { IsSpread bool; Key any; Value any; Map OrderedMap }\n\nfunc Unit(value any) struct{} { return struct{}{} }\nfunc Int(value any) int64 { return value.(int64) }\nfunc Float(value any) float64 { return value.(float64) }\nfunc Bool(value any) bool { return value.(bool) }\nfunc String(value any) string { return value.(string) }\nfunc List(value any) []any { return value.([]any) }\nfunc Record(value any) map[string]any { return value.(map[string]any) }\nfunc AsOption(value any) Option { return value.(Option) }\nfunc AsUnion(value any) Union { return value.(Union) }\nfunc AsMap(value any) OrderedMap { return value.(OrderedMap) }\nfunc Some(value any) Option { return Option{HasValue: true, Value: value} }\nfunc None() Option { return Option{} }\nfunc IsSome(value any) bool { return AsOption(value).HasValue }\nfunc OptionValue(value any) any { return AsOption(value).Value }\nfunc MakeUnion(typeName string, caseName string) Union { return Union{TypeName: typeName, CaseName: caseName} }\nfunc MakeUnionValue(typeName string, caseName string, value any) Union { return Union{TypeName: typeName, CaseName: caseName, HasValue: true, Value: value} }\nfunc IsUnion(value any, typeName string, caseName string) bool { item := AsUnion(value); return item.TypeName == typeName && item.CaseName == caseName }\nfunc UnionHasValue(value any) bool { return AsUnion(value).HasValue }\nfunc UnionValue(value any) any { return AsUnion(value).Value }\nfunc HasField(value any, name string) bool { _, ok := Record(value)[name]; return ok }\nfunc Equal(left any, right any) bool { return reflect.DeepEqual(left, right) }\nfunc Cons(head any, tail []any) []any { return append([]any{head}, tail...) }\nfunc Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) }\nfunc Tail(value any) []any { return append([]any{}, List(value)[1:]...) }\nfunc Range(first int64, last int64) []any { step := int64(1); if first > last { step = -1 }; values := []any{}; for current := first; ; current += step { values = append(values, current); if current == last { break } }; return values }\nfunc UpdateRecord(value any, updates map[string]any) map[string]any { result := map[string]any{}; for key, item := range Record(value) { result[key] = item }; for key, item := range updates { result[key] = item }; return result }\nfunc Pair(key any, value any) MapPart { return MapPart{Key: key, Value: value} }\nfunc SpreadMap(value any) MapPart { return MapPart{IsSpread: true, Map: AsMap(value)} }\nfunc mapSet(entries []MapEntry, key any, value any, replace bool) []MapEntry { for index, entry := range entries { if Equal(entry.Key, key) { if replace { entries[index].Value = value }; return entries } }; return append(entries, MapEntry{Key: key, Value: value}) }\nfunc BuildMap(parts []MapPart) OrderedMap { entries := []MapEntry{}; for _, part := range parts { if part.IsSpread { for _, entry := range part.Map.Entries { entries = mapSet(entries, entry.Key, entry.Value, false) } } else { entries = mapSet(entries, part.Key, part.Value, true) } }; sort.SliceStable(entries, func(left int, right int) bool { return Format(entries[left].Key) < Format(entries[right].Key) }); return OrderedMap{Entries: entries} }\nfunc Index(value any, index any) Option { switch target := value.(type) { case []any: position := Int(index); if position < 0 || position >= int64(len(target)) { return None() }; return Some(target[position]); case OrderedMap: for _, entry := range target.Entries { if Equal(entry.Key, index) { return Some(entry.Value) } }; return None(); default: panic(\"Index access requires a list or map\") } }\nfunc Intrinsic(name string, args []any) any { switch name { case \"ignore\": return struct{}{}; case \"List.empty\": return []any{}; case \"Map.empty\": return OrderedMap{}; case \"Int.tryParse\": value, error := strconv.ParseInt(String(args[0]), 10, 64); if error != nil { return None() }; return Some(value); case \"Float.tryParse\": value, error := strconv.ParseFloat(String(args[0]), 64); if error != nil { return None() }; return Some(value); case \"Bool.tryParse\": value, error := strconv.ParseBool(String(args[0])); if error != nil { return None() }; return Some(value); case \"Int.toString\", \"Float.toString\", \"Bool.toString\": return Format(args[0]); case \"String.replace\": return strings.ReplaceAll(String(args[2]), String(args[0]), String(args[1])); case \"String.indexOf\": index := strings.Index(String(args[1]), String(args[0])); if index < 0 { return None() }; return Some(int64(index)); case \"String.toLower\": return strings.ToLower(String(args[0])); case \"String.toUpper\": return strings.ToUpper(String(args[0])); case \"String.substring\": start, length, value := Int(args[0]), Int(args[1]), String(args[2]); if start < 0 || length < 0 || start + length > int64(len(value)) { return None() }; return Some(value[start:start+length]); case \"String.concat\": values := List(args[1]); parts := make([]string, len(values)); for index, value := range values { parts[index] = String(value) }; return strings.Join(parts, String(args[0])); case \"String.split\": values := strings.Split(String(args[1]), String(args[0])); result := make([]any, len(values)); for index, value := range values { result[index] = value }; return result; case \"String.endsWith\": return strings.HasSuffix(String(args[1]), String(args[0])); case \"List.length\": return int64(len(List(args[0]))); case \"List.rev\": values := List(args[0]); result := make([]any, len(values)); for index, value := range values { result[len(values)-index-1] = value }; return result; case \"List.tryHead\": values := List(args[0]); if len(values) == 0 { return None() }; return Some(values[0]); case \"List.tail\": return Tail(args[0]); case \"List.append\": return Append(List(args[0]), List(args[1])); case \"List.contains\": for _, value := range List(args[1]) { if Equal(args[0], value) { return true } }; return false; case \"Option.isNone\": return !IsSome(args[0]); case \"Option.isSome\": return IsSome(args[0]); case \"Option.defaultValue\": option := AsOption(args[1]); if option.HasValue { return option.Value }; return args[0]; case \"Map.count\": return int64(len(AsMap(args[0]).Entries)); case \"Map.containsKey\": return IsSome(Index(args[1], args[0])); case \"Map.tryGet\": return Index(args[1], args[0]); default: panic(\"Unsupported FScript intrinsic: \" + name) } }\nfunc Format(value any) string { switch item := value.(type) { case bool: if item { return \"true\" }; return \"false\"; case Option: if !item.HasValue { return \"None\" }; return \"Some \" + Format(item.Value); case Union: if !item.HasValue { return item.CaseName }; return item.CaseName + \" \" + Format(item.Value); case OrderedMap: parts := make([]string, len(item.Entries)); for index, entry := range item.Entries { parts[index] = \"[\" + Format(entry.Key) + \"] = \" + Format(entry.Value) }; return \"{ \" + strings.Join(parts, \"; \") + \" }\"; case []any: parts := make([]string, len(item)); for index, value := range item { parts[index] = Format(value) }; return \"[\" + strings.Join(parts, \";\") + \"]\"; default: return fmt.Sprint(value) } }\nfunc Interpolate(values []any) string { var result strings.Builder; for _, value := range values { result.WriteString(Format(value)) }; return result.String() }\nfunc Fail(value any) { panic(fmt.Sprint(value)) }\n" } ] with ex -> Error [ { Code = "FSB4002"; Stage = GoGeneration; Message = ex.Message; Span = None } ] diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs index 30395f1..5c982ed 100644 --- a/src/FScript.Compiler/Hir.fs +++ b/src/FScript.Compiler/Hir.fs @@ -86,9 +86,11 @@ module Hir = | ConsPattern of Pattern * Pattern | TuplePattern of Pattern list | RecordPattern of (string * Pattern) list + | MapPattern of (Pattern * Pattern) list * Pattern option | SomePattern of Pattern | NonePattern | UnionPattern of string * string * Pattern option + | TypePattern of Type and MatchCase = { Pattern: Pattern diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index 6f4c82b..a76fa71 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -85,6 +85,24 @@ module HirLowering = let fields, bindingMaps = List.unzip values let bindings = bindingMaps |> List.fold (Map.fold (fun env key value -> Map.add key value env)) Map.empty make (Hir.RecordPattern fields), bindings) + | PMap(clauses, tail, _) -> + let keyType, valueType = match expected with TMap(key, value) -> key, value | _ -> Types.freshVar(), Types.freshVar() + let loweredClauses = + clauses |> List.map (fun (keyPattern, valuePattern) -> + match lowerPattern state keyType keyPattern, lowerPattern state valueType valuePattern with + | Ok(key, keyBindings), Ok(value, valueBindings) -> + Ok((key, value), Map.fold (fun env name id -> Map.add name id env) keyBindings valueBindings) + | Error a, Error b -> Error(a @ b) | Error errors, _ | _, Error errors -> Error errors) + |> collectResults + let loweredTail = + tail |> Option.map (lowerPattern state expected >> Result.map Some) |> Option.defaultValue (Ok None) + match loweredClauses, loweredTail with + | Ok clausesWithBindings, Ok tailValue -> + let clauses, bindingMaps = List.unzip clausesWithBindings + let tailPattern, tailBindings = tailValue |> Option.map (fun (pattern, bindings) -> Some pattern, bindings) |> Option.defaultValue (None, Map.empty) + let bindings = tailBindings :: bindingMaps |> List.fold (Map.fold (fun env name id -> Map.add name id env)) Map.empty + Ok(make (Hir.MapPattern(clauses, tailPattern)), bindings) + | Error a, Error b -> Error(a @ b) | Error errors, _ | _, Error errors -> Error errors | PUnionCase(typeName, caseName, payload, _) -> let resolvedTypeName, payloadType = match expected with @@ -96,7 +114,7 @@ module HirLowering = | None, _ -> Ok(make (Hir.UnionPattern(resolvedTypeName, caseName, None)), Map.empty) | Some item, Some itemType -> lowerPattern state itemType item |> Result.map (fun (value, bindings) -> make (Hir.UnionPattern(resolvedTypeName, caseName, Some value)), bindings) | Some item, None -> lowerPattern state (Types.freshVar()) item |> Result.map (fun (value, bindings) -> make (Hir.UnionPattern(resolvedTypeName, caseName, Some value)), bindings) - | _ -> Error [ Diagnostic.hir "FSH1004" "Pattern form is not yet supported by HIR lowering." (Some span) ] + | PTypeRef _ -> Ok(make (Hir.TypePattern expected), Map.empty) let rec private lowerExpression state environment fallback expression = let span = Ast.spanOfExpr expression diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index ef725da..d69d3c3 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -40,8 +40,12 @@ module HirValidation = | Hir.ConsPattern(head, tail) -> Set.union (bindings head) (bindings tail) | Hir.TuplePattern items -> items |> List.map bindings |> Set.unionMany | Hir.RecordPattern fields -> fields |> List.map (snd >> bindings) |> Set.unionMany + | Hir.MapPattern(clauses, tail) -> + let clauseBindings = clauses |> List.collect (fun (key, value) -> [ bindings key; bindings value ]) + tail |> Option.map bindings |> Option.toList |> (@) clauseBindings |> Set.unionMany | Hir.SomePattern item -> bindings item | Hir.UnionPattern(_, _, payload) -> payload |> Option.map bindings |> Option.defaultValue Set.empty + | Hir.TypePattern _ -> Set.empty | _ -> Set.empty let caseReferences (case: Hir.MatchCase) = let bound = bindings case.Pattern diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index 82d876e..4b71f79 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -91,9 +91,17 @@ module IrArtifact = for name, item in fields do writer.WriteStartObject(); writer.WriteString("name", name); writer.WritePropertyName("pattern"); writePattern writer item; writer.WriteEndObject() writer.WriteEndArray() + | Lir.MapPattern(clauses, tail) -> + writer.WriteString("kind", "map"); writer.WriteStartArray("clauses") + for key, item in clauses do + writer.WriteStartObject(); writer.WritePropertyName("key"); writePattern writer key + writer.WritePropertyName("value"); writePattern writer item; writer.WriteEndObject() + writer.WriteEndArray() + match tail with Some item -> writer.WritePropertyName("tail"); writePattern writer item | None -> writer.WriteNull("tail") | Lir.UnionPattern(typeName, caseName, payload) -> writer.WriteString("kind", "union"); writer.WriteString("typeName", typeName); writer.WriteString("caseName", caseName) match payload with Some item -> writer.WritePropertyName("payload"); writePattern writer item | None -> writer.WriteNull("payload") + | Lir.TypePattern patternType -> writer.WriteString("kind", "type"); writer.WritePropertyName("patternType"); writeType writer patternType writer.WriteEndObject() let rec private writeExpression (writer: Utf8JsonWriter) (value: Lir.Expression) = @@ -271,9 +279,14 @@ module IrArtifact = | "cons" -> Lir.ConsPattern(readPattern (property "head" element), readPattern (property "tail" element)) | "tuple" -> Lir.TuplePattern((property "items" element).EnumerateArray() |> Seq.map readPattern |> Seq.toList) | "record" -> Lir.RecordPattern((property "fields" element).EnumerateArray() |> Seq.map (fun field -> stringProperty "name" field, readPattern (property "pattern" field)) |> Seq.toList) + | "map" -> + let clauses = (property "clauses" element).EnumerateArray() |> Seq.map (fun clause -> readPattern (property "key" clause), readPattern (property "value" clause)) |> Seq.toList + let tail = property "tail" element + Lir.MapPattern(clauses, if tail.ValueKind = JsonValueKind.Null then None else Some(readPattern tail)) | "union" -> let payload = property "payload" element Lir.UnionPattern(stringProperty "typeName" element, stringProperty "caseName" element, if payload.ValueKind = JsonValueKind.Null then None else Some(readPattern payload)) + | "type" -> Lir.TypePattern(readType (property "patternType" element)) | kind -> invalidOp $"Unknown LIR pattern kind '{kind}'." { Type = readType (property "type" element); Span = readSpan (property "span" element); Node = node } diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index 35ebc96..0604741 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -74,9 +74,11 @@ module Lir = | ConsPattern of Pattern * Pattern | TuplePattern of Pattern list | RecordPattern of (string * Pattern) list + | MapPattern of (Pattern * Pattern) list * Pattern option | SomePattern of Pattern | NonePattern | UnionPattern of string * string * Pattern option + | TypePattern of Type and DecisionCase = { Pattern: Pattern @@ -127,9 +129,11 @@ module LirLowering = | Hir.ConsPattern(head, tail) -> Lir.ConsPattern(pattern head, pattern tail) | Hir.TuplePattern items -> Lir.TuplePattern(List.map pattern items) | Hir.RecordPattern fields -> Lir.RecordPattern(fields |> List.map (fun (name, item) -> name, pattern item)) + | Hir.MapPattern(clauses, tail) -> Lir.MapPattern(clauses |> List.map (fun (key, value) -> pattern key, pattern value), Option.map pattern tail) | Hir.SomePattern item -> Lir.SomePattern(pattern item) | Hir.NonePattern -> Lir.NonePattern | Hir.UnionPattern(typeName, caseName, payload) -> Lir.UnionPattern(typeName, caseName, Option.map pattern payload) + | Hir.TypePattern patternType -> Lir.TypePattern patternType { Type = value.Type; Span = value.Span; Node = node } let rec private expression (value: Hir.Expression) : Lir.Expression = diff --git a/src/FScript.Interpreter.Lir/LirInterpreter.fs b/src/FScript.Interpreter.Lir/LirInterpreter.fs index c8d48a1..9dd482f 100644 --- a/src/FScript.Interpreter.Lir/LirInterpreter.fs +++ b/src/FScript.Interpreter.Lir/LirInterpreter.fs @@ -174,8 +174,27 @@ module LirInterpreter = let rec private matchPattern (pattern: Lir.Pattern) value = let combine left right = match left, right with - | Some a, Some b -> Some(Map.fold (fun env key item -> Map.add key item env) a b) + | Some a, Some b -> + b |> Map.fold (fun state key item -> + state |> Option.bind (fun env -> + match env |> Map.tryFind key with + | Some existing when not (equal existing item) -> None + | _ -> Some(Map.add key item env))) (Some a) | _ -> None + let rec matchesType expected item = + match expected, item with + | TUnit, VUnit | TInt, VInt _ | TFloat, VFloat _ | TBool, VBool _ | TString, VString _ | TTypeToken, VTypeToken _ -> true + | TTuple types, VTuple values -> types.Length = values.Length && List.forall2 matchesType types values + | TList itemType, VList values -> values |> List.forall (matchesType itemType) + | TOption _, VOption None -> true + | TOption itemType, VOption(Some value) -> matchesType itemType value + | TRecord fields, VRecord values -> fields |> Map.forall (fun name fieldType -> values |> Map.tryFind name |> Option.exists (matchesType fieldType)) + | TMap(keyType, valueType), VMap entries -> entries |> List.forall (fun (key, value) -> matchesType keyType key && matchesType valueType value) + | TUnion(typeName, _), VUnion(valueType, _, _) | TNamed typeName, VUnion(valueType, _, _) -> typeName = valueType + | TNamed _, VRecord _ -> true + | TFun _, (VClosure _ | VIntrinsic _ | VExternal _) -> true + | TVar _, _ -> true + | _ -> false match pattern.Node, value with | Lir.Wildcard, _ -> Some Map.empty | Lir.Binding(id, _), item -> Some(Map.ofList [ id, item ]) @@ -188,10 +207,26 @@ module LirInterpreter = | Lir.RecordPattern patterns, VRecord values -> patterns |> List.fold (fun state (name, itemPattern) -> combine state (values |> Map.tryFind name |> Option.bind (matchPattern itemPattern))) (Some Map.empty) + | Lir.MapPattern(clauses, tailPattern), VMap entries -> + let rec matchClauses bindings remaining = function + | [] -> + match tailPattern with + | Some tail -> combine (Some bindings) (matchPattern tail (VMap remaining)) + | None when not clauses.IsEmpty || remaining.IsEmpty -> Some bindings + | None -> None + | (keyPattern, valuePattern) :: rest -> + remaining |> List.indexed |> List.tryPick (fun (index, (key, item)) -> + combine (Some bindings) (matchPattern keyPattern key) + |> Option.bind (fun withKey -> combine (Some withKey) (matchPattern valuePattern item)) + |> Option.bind (fun withValue -> + let next = remaining |> List.indexed |> List.choose (fun (position, entry) -> if position = index then None else Some entry) + matchClauses withValue next rest)) + matchClauses Map.empty entries clauses | Lir.SomePattern itemPattern, VOption(Some item) -> matchPattern itemPattern item | Lir.NonePattern, VOption None -> Some Map.empty | Lir.UnionPattern(typeName, caseName, payloadPattern), VUnion(valueType, valueCase, payload) when typeName = valueType && caseName = valueCase -> match payloadPattern, payload with None, None -> Some Map.empty | Some pattern, Some value -> matchPattern pattern value | _ -> None + | Lir.TypePattern expected, item when matchesType expected item -> Some Map.empty | _ -> None let rec private evaluate environment (expression: Lir.Expression) = diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index 7e5f6c0..3de1624 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -48,6 +48,8 @@ type GoBackendTests() = [] [] [] + [ value\n | {} -> 0", "1")>] + [ true\n | _ -> false", "true")>] member _.``Generated projects run with Go`` (source: string, expected: string) = let input = { Program = Parser.parseProgram source; Externs = [] } let compiled = diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index 730ac8f..310ff81 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -50,7 +50,9 @@ type IrArtifactTests() = "let sum pair = (let (left, right) = pair\n left + right\n)\nsum (1, 2)" "for value in [1; 2; 3] do ignore value" "type Package = { Name: string }\ntypeof Package" - "(let rec even n = if n = 0 then true else odd (n - 1)\nand odd n = if n = 0 then false else even (n - 1)\neven 10\n)" ] do + "(let rec even n = if n = 0 then true else odd (n - 1)\nand odd n = if n = 0 then false else even (n - 1)\neven 10\n)" + "match { [\"a\"] = 1 } with\n | { [key] = value; ..tail } -> value\n | {} -> 0" + "match {| Name = \"Ada\" |} with\n | {| Name: string |} -> true\n | _ -> false" ] do let bytes = compile source |> _.Lir |> IrArtifact.serialize match IrArtifact.deserialize bytes with | Ok _ -> () diff --git a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs index 5c18fe8..c819656 100644 --- a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs +++ b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs @@ -198,3 +198,13 @@ type LirInterpreterTests() = match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VBool true)) | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [ (key, value, Map.count tail)\n | {} -> (\"\", 0, 0)")>] + [ (\"Name\", 1, 0)\n | _ -> (\"\", 0, 0)")>] + member _.``Executes map and type-reference patterns`` (source: string) = + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok _ -> Assert.Pass() + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) From feeb1a0353f75d85dd34de35bf18e92372244e8a Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 17:47:14 +0200 Subject: [PATCH 024/133] refactor(language): resolve typed symbol references --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 + docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Language/TypeInfer.fs | 149 +++++++++++++++++- src/FScript.Language/TypedAst.fs | 68 -------- .../ResolvedInferenceTests.fs | 22 +++ 6 files changed, 174 insertions(+), 69 deletions(-) delete mode 100644 src/FScript.Language/TypedAst.fs diff --git a/CHANGELOG.md b/CHANGELOG.md index cf452d7..efcfa2d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Exposed deterministic resolved frontend symbols, normalized declaration schemes, and instantiated lexical reference types without changing legacy inference APIs. - Added deterministic ordered-map and runtime type-reference patterns to HIR, artifacts, LIR execution, and generated Go. - Added serializable local mutual-recursive closure groups to HIR, LIR execution, and generated Go projects. - Preserved source-order execution for every top-level expression and added compiled top-level pattern bindings. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 14cd914..84ada03 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -12,6 +12,8 @@ source -> parser/includes -> inference + resolved typed tree -> HIR -> LIR `FScript.Language` remains the parser, type-system, legacy evaluator, and Fable-compatible package. It emits a resolved typed tree in the same inference pass as its compatibility `TypedProgram`. +Resolved inference assigns deterministic program-local IDs to builtins, externs, types, union cases, declarations, parameters, pattern bindings, and locals in normalized source traversal. Declaration schemes and instantiated reference types normalize type variables from zero, so results do not depend on the process-global fresh-variable counter. The legacy `TypedProgram` remains a projection with unchanged public entry points. + `FScript.Compiler` is a reusable, initially non-packaged .NET assembly. It owns HIR, HIR validation, lowering, LIR, LIR validation, and artifact serialization. It depends on `FScript.Language`; the language assembly never depends on the compiler. Validated LIR consumers are separate leaf assemblies. `FScript.Interpreter.Lir` owns execution and runtime LIR values. `FScript.Backend.Go` owns representation selection, Go emission, and embedded runtime templates. Both reference `FScript.Compiler`, while the compiler references neither consumer. Public modules retain the `FScript.Compiler` namespace so the API surface remains source-compatible while assembly ownership is explicit. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index f038d45..ad942d7 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -9,6 +9,7 @@ This document records the behavior that the typed IR, LIR interpreter, and compi - Includes are resolved into one flattened program before inference. Alias-qualified declarations are rewritten by the include resolver; unresolved `SImport` nodes are rejected by inference. - Hindley-Milner inference generalizes top-level and local let bindings after applying substitutions. Recursive bindings are inferred monomorphically inside their recursive group and generalized afterward. - The active typed program stores top-level types but retains parser expressions in bodies. Compiler lowering therefore requires a new resolved, recursively typed result emitted by the inference pass; types cannot safely be reconstructed afterward. +- Resolved inference exposes deterministic symbols for every lexical binding category and links each captured reference to its target ID with its instantiated type. Type-variable numbering in resolved schemes and reference types is normalized independently of prior inference calls. - Top-level statements execute in source order. Every non-final expression must have type `unit`; the final expression is the program result. - Compiled top-level expressions execute exactly once in source order even when interleaved with declarations. Top-level pattern bindings evaluate their scrutinee once, fail before later statements when unmatched, and expose all bound names to subsequent statements. diff --git a/src/FScript.Language/TypeInfer.fs b/src/FScript.Language/TypeInfer.fs index 4890ce9..187a399 100644 --- a/src/FScript.Language/TypeInfer.fs +++ b/src/FScript.Language/TypeInfer.fs @@ -167,8 +167,26 @@ module TypeInfer = type ResolvedSymbolReference = { SymbolId: int Name: string + Type: Type Span: Span } + type ResolvedSymbolKind = + | TypeSymbol + | DeclarationSymbol + | ParameterSymbol + | PatternBindingSymbol + | LocalSymbol + | BuiltinSymbol + | ExternSymbol + | UnionCaseSymbol + + type ResolvedSymbol = + { SymbolId: int + Name: string + Kind: ResolvedSymbolKind + Span: Span + Scheme: Scheme option } + type private LocalVariableTypeCapture = { Name: string Span: Span @@ -184,6 +202,7 @@ module TypeInfer = type ResolvedProgram = { TypedProgram: TypedProgram ExpressionTypes: ResolvedExpressionType list + Symbols: ResolvedSymbol list SymbolReferences: ResolvedSymbolReference list } type private InferenceTelemetry = @@ -1445,9 +1464,137 @@ module TypeInfer = let inferResolvedProgramWithExterns (externs: ExternalFunction list) (program: Program) : ResolvedProgram = let typed, _, expressions = inferProgramWithExternsRawAndLocals true externs program + let expressionTypes = expressions |> List.fold (fun state item -> Map.add item.Expr item.Type state) Map.empty + let mutable nextSymbol = 0 + let symbols = ResizeArray() + let references = ResizeArray() + let normalizeScheme (Forall(variables, schemeType)) = + let mapping = variables |> List.mapi (fun index variable -> variable, index) |> Map.ofList + let rec normalize = function + | TVar variable -> TVar(mapping |> Map.tryFind variable |> Option.defaultValue variable) + | TList item -> TList(normalize item) | TTask item -> TTask(normalize item) | TOption item -> TOption(normalize item) + | TTuple items -> TTuple(List.map normalize items) + | TRecord fields -> TRecord(Map.map (fun _ value -> normalize value) fields) + | TMap(key, value) -> TMap(normalize key, normalize value) + | TFun(argument, result) -> TFun(normalize argument, normalize result) + | TUnion(name, cases) -> TUnion(name, Map.map (fun _ payload -> Option.map normalize payload) cases) + | value -> value + Forall([ 0 .. variables.Length - 1 ], normalize schemeType) + let normalizeTypeVariables value = + let mutable next = 0 + let mutable mapping = Map.empty + let rec normalize = function + | TVar variable -> + match Map.tryFind variable mapping with + | Some normalized -> TVar normalized + | None -> + let normalized = next + next <- next + 1 + mapping <- Map.add variable normalized mapping + TVar normalized + | TList item -> TList(normalize item) | TTask item -> TTask(normalize item) | TOption item -> TOption(normalize item) + | TTuple items -> TTuple(List.map normalize items) + | TRecord fields -> TRecord(Map.map (fun _ item -> normalize item) fields) + | TMap(key, item) -> TMap(normalize key, normalize item) + | TFun(argument, result) -> TFun(normalize argument, normalize result) + | TUnion(name, cases) -> TUnion(name, Map.map (fun _ payload -> Option.map normalize payload) cases) + | item -> item + normalize value + let allocate name kind span scheme = + let id = nextSymbol + nextSymbol <- nextSymbol + 1 + symbols.Add { SymbolId = id; Name = name; Kind = kind; Span = span; Scheme = Option.map normalizeScheme scheme } + id + let syntheticSpan = Span.mk (Span.pos 0 0) (Span.pos 0 0) + let mutable globalEnvironment = Map.empty + for name, scheme in BuiltinSignatures.builtinSchemes |> Map.toList |> List.sortBy fst do + globalEnvironment <- Map.add name (allocate name BuiltinSymbol syntheticSpan (Some scheme)) globalEnvironment + for external in externs |> List.sortBy _.Name do + globalEnvironment <- Map.add external.Name (allocate external.Name ExternSymbol syntheticSpan (Some external.Scheme)) globalEnvironment + for statement in typed do + match statement with + | TSType definition -> + globalEnvironment <- Map.add definition.Name (allocate definition.Name TypeSymbol definition.Span None) globalEnvironment + for caseName, _ in definition.Cases do + let id = allocate $"{definition.Name}.{caseName}" UnionCaseSymbol definition.Span None + globalEnvironment <- globalEnvironment |> Map.add caseName id |> Map.add $"{definition.Name}.{caseName}" id + | TSLet(name, _, inferred, _, _, span) -> + globalEnvironment <- Map.add name (allocate name DeclarationSymbol span (Some(Types.generalize Map.empty inferred))) globalEnvironment + | TSLetPattern(_, _, bindings, _, span) -> + for name, inferred in bindings |> Map.toList do + globalEnvironment <- Map.add name (allocate name DeclarationSymbol span (Some(Types.generalize Map.empty inferred))) globalEnvironment + | TSLetRecGroup(bindings, _, _) -> + for name, _, inferred, span in bindings do + globalEnvironment <- Map.add name (allocate name DeclarationSymbol span (Some(Types.generalize Map.empty inferred))) globalEnvironment + | TSExpr _ -> () + let recordReference environment name expression span = + environment |> Map.tryFind name |> Option.iter (fun id -> + references.Add { SymbolId = id; Name = name; Type = expressionTypes |> Map.tryFind expression |> Option.defaultValue TUnit |> normalizeTypeVariables; Span = span }) + let rec bindPattern kind environment pattern = + match pattern with + | PVar(name, span) -> Map.add name (allocate name kind span None) environment + | PCons(head, tail, _) -> bindPattern kind (bindPattern kind environment head) tail + | PTuple(items, _) -> items |> List.fold (bindPattern kind) environment + | PRecord(fields, _) -> fields |> List.fold (fun env (_, item) -> bindPattern kind env item) environment + | PMap(clauses, tail, _) -> + let withClauses = clauses |> List.fold (fun env (key, value) -> bindPattern kind (bindPattern kind env key) value) environment + tail |> Option.map (bindPattern kind withClauses) |> Option.defaultValue withClauses + | PSome(item, _) -> bindPattern kind environment item + | PUnionCase(_, _, payload, _) -> payload |> Option.map (bindPattern kind environment) |> Option.defaultValue environment + | _ -> environment + let rec visit environment expression = + let visitSame item = visit environment item + match expression with + | EVar(name, span) -> recordReference environment name expression span + | ENameOf(name, span) -> recordReference environment name expression span + | ETypeOf(name, span) -> recordReference environment name expression span + | ELambda(parameter, body, _) -> + let id = allocate parameter.Name ParameterSymbol parameter.Span None + visit (Map.add parameter.Name id environment) body + | EApply(fn, argument, _) | EBinOp(_, fn, argument, _) | EAppend(fn, argument, _) | ECons(fn, argument, _) | EIndexGet(fn, argument, _) -> visitSame fn; visitSame argument + | EIf(condition, whenTrue, whenFalse, _) -> visitSame condition; visitSame whenTrue; visitSame whenFalse + | ELet(name, binding, body, recursive, _, span) -> + let id = allocate name LocalSymbol span None + visit (if recursive then Map.add name id environment else environment) binding + visit (Map.add name id environment) body + | ELetPattern(pattern, binding, body, _) -> visitSame binding; visit (bindPattern PatternBindingSymbol environment pattern) body + | ELetRecGroup(bindings, body, _) -> + let recursiveEnvironment = bindings |> List.fold (fun env (name, _, _, _, span) -> Map.add name (allocate name LocalSymbol span None) env) environment + for _, parameters, _, value, span in bindings do + let folded = Seq.foldBack (fun parameter inner -> ELambda(parameter, inner, span)) parameters value + visit recursiveEnvironment folded + visit recursiveEnvironment body + | EMatch(target, cases, _) -> + visitSame target + for pattern, guard, body, _ in cases do + let caseEnvironment = bindPattern PatternBindingSymbol environment pattern + Option.iter (visit caseEnvironment) guard + visit caseEnvironment body + | EFor(name, source, body, span) -> + visitSame source + let id = allocate name LocalSymbol span None + visit (Map.add name id environment) body + | EList(items, _) | ETuple(items, _) -> List.iter visitSame items + | ERecord(fields, _) | EStructuralRecord(fields, _) -> fields |> List.iter (snd >> visitSame) + | EMap(entries, _) -> entries |> List.iter (function MEKeyValue(key, value) -> visitSame key; visitSame value | MESpread value -> visitSame value) + | ERecordUpdate(target, fields, _) | EStructuralRecordUpdate(target, fields, _) -> visitSame target; fields |> List.iter (snd >> visitSame) + | EFieldGet(EVar(moduleName, _), fieldName, span) when Map.containsKey $"{moduleName}.{fieldName}" environment -> recordReference environment $"{moduleName}.{fieldName}" expression span + | EFieldGet(target, _, _) -> visitSame target + | ESome(item, _) | EParen(item, _) | ERaise(item, _) -> visitSame item + | ERange(first, last, _) -> visitSame first; visitSame last + | EInterpolatedString(parts, _) -> parts |> List.iter (function IPText _ -> () | IPExpr item -> visitSame item) + | EUnit _ | ELiteral _ | ENone _ -> () + for statement in typed do + match statement with + | TSLet(_, expression, _, _, _, _) -> visit globalEnvironment expression + | TSLetPattern(pattern, expression, _, _, _) -> visit globalEnvironment expression; bindPattern PatternBindingSymbol globalEnvironment pattern |> ignore + | TSLetRecGroup(bindings, _, _) -> bindings |> List.iter (fun (_, expression, _, _) -> visit globalEnvironment expression) + | TSExpr expression -> visit globalEnvironment expression.Expr + | TSType _ -> () { TypedProgram = typed ExpressionTypes = expressions - SymbolReferences = [] } + Symbols = symbols |> Seq.toList + SymbolReferences = references |> Seq.toList } let inferResolvedProgram (program: Program) : ResolvedProgram = inferResolvedProgramWithExterns [] program diff --git a/src/FScript.Language/TypedAst.fs b/src/FScript.Language/TypedAst.fs deleted file mode 100644 index 2e8e874..0000000 --- a/src/FScript.Language/TypedAst.fs +++ /dev/null @@ -1,68 +0,0 @@ -namespace FScript.Language - -type TypedExpr = - | TELiteral of Literal * Type * Span - | TEVar of string * Type * Span - | TELambda of string * TypedExpr * Type * Span - | TEApply of TypedExpr * TypedExpr * Type * Span - | TEIf of TypedExpr * TypedExpr * TypedExpr * Type * Span - | TEMatch of TypedExpr * (Pattern * TypedExpr option * TypedExpr * Span) list * Type * Span - | TELet of string * TypedExpr * TypedExpr * bool * Type * Span - | TEList of TypedExpr list * Type * Span - | TETuple of TypedExpr list * Type * Span - | TERecord of (string * TypedExpr) list * Type * Span - | TEFieldGet of TypedExpr * string * Type * Span - | TEConstructor of string * string * Type option * Type * Span - | TECons of TypedExpr * TypedExpr * Type * Span - | TEAppend of TypedExpr * TypedExpr * Type * Span - | TEBinOp of string * TypedExpr * TypedExpr * Type * Span - | TESome of TypedExpr * Type * Span - | TENone of Type * Span - -and TypedStmt = - | TSLet of string * string list * TypedExpr * bool * Span - | TSType of TypeDef - | TSExpr of TypedExpr - -type TypedProgram = TypedStmt list - -module TypedAst = - let spanOfExpr expr = - match expr with - | TELiteral (_, _, s) -> s - | TEVar (_, _, s) -> s - | TELambda (_, _, _, s) -> s - | TEApply (_, _, _, s) -> s - | TEIf (_, _, _, _, s) -> s - | TEMatch (_, _, _, s) -> s - | TELet (_, _, _, _, _, s) -> s - | TEList (_, _, s) -> s - | TETuple (_, _, s) -> s - | TERecord (_, _, s) -> s - | TEFieldGet (_, _, _, s) -> s - | TEConstructor (_, _, _, _, s) -> s - | TECons (_, _, _, s) -> s - | TEAppend (_, _, _, s) -> s - | TEBinOp (_, _, _, _, s) -> s - | TESome (_, _, s) -> s - | TENone (_, s) -> s - - let typeOf expr = - match expr with - | TELiteral (_, t, _) -> t - | TEVar (_, t, _) -> t - | TELambda (_, _, t, _) -> t - | TEApply (_, _, t, _) -> t - | TEIf (_, _, _, t, _) -> t - | TEMatch (_, _, t, _) -> t - | TELet (_, _, _, _, t, _) -> t - | TEList (_, t, _) -> t - | TETuple (_, t, _) -> t - | TERecord (_, t, _) -> t - | TEFieldGet (_, _, t, _) -> t - | TEConstructor (_, _, _, t, _) -> t - | TECons (_, _, t, _) -> t - | TEAppend (_, _, t, _) -> t - | TEBinOp (_, _, _, t, _) -> t - | TESome (_, t, _) -> t - | TENone (t, _) -> t diff --git a/tests/FScript.Language.Tests/ResolvedInferenceTests.fs b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs index 2c55ae8..31ff6bd 100644 --- a/tests/FScript.Language.Tests/ResolvedInferenceTests.fs +++ b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs @@ -29,3 +29,25 @@ type ResolvedInferenceTests() = let resolved = TypeInfer.inferResolvedProgram program Assert.That(resolved.ExpressionTypes |> List.exists (fun item -> item.Type = TInt), Is.True) + + [] + member _.``Resolved inference assigns deterministic semantic symbols and typed references`` () = + let program = Helpers.parse "let apply f value = f value\napply (fun item -> item + 1) 41" + let first = TypeInfer.inferResolvedProgram program + let second = TypeInfer.inferResolvedProgram program + + Assert.That(first.Symbols = second.Symbols, Is.True) + Assert.That(first.SymbolReferences = second.SymbolReferences, Is.True) + Assert.That((first.Symbols |> List.map _.SymbolId) = [ 0 .. first.Symbols.Length - 1 ], Is.True) + Assert.That(first.Symbols |> List.exists (fun symbol -> symbol.Name = "apply" && symbol.Kind = TypeInfer.DeclarationSymbol), Is.True) + Assert.That(first.Symbols |> List.exists (fun symbol -> symbol.Name = "item" && symbol.Kind = TypeInfer.ParameterSymbol), Is.True) + Assert.That(first.SymbolReferences |> List.exists (fun reference -> reference.Name = "apply" && reference.Type = TFun(TFun(TInt, TInt), TFun(TInt, TInt))), Is.True) + + [] + member _.``Resolved declaration schemes normalize type variables`` () = + let resolved = Helpers.parse "let identity value = value\nidentity 1" |> TypeInfer.inferResolvedProgram + let scheme = + resolved.Symbols + |> List.find (fun symbol -> symbol.Name = "identity" && symbol.Kind = TypeInfer.DeclarationSymbol) + |> _.Scheme + Assert.That(scheme, Is.EqualTo(Some(Forall([ 0 ], TFun(TVar 0, TVar 0))))) From 8d4ae464288f26a7c5ac40aaef833c7f3837ddd6 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 17:50:41 +0200 Subject: [PATCH 025/133] feat(compiler): harden versioned IR metadata --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 + docs/specs/compiler-semantics-baseline.md | 2 + src/FScript.Compiler/IrArtifact.fs | 74 ++++++++++++++++--- .../FScript.Compiler.Tests/IrArtifactTests.fs | 40 ++++++++++ 5 files changed, 107 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index efcfa2d..42b69ff 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added versioned `.fsir` compiler/language metadata, normalized source SHA-256 records, ordered dependencies, and strict metadata validation. - Exposed deterministic resolved frontend symbols, normalized declaration schemes, and instantiated lexical reference types without changing legacy inference APIs. - Added deterministic ordered-map and runtime type-reference patterns to HIR, artifacts, LIR execution, and generated Go. - Added serializable local mutual-recursive closure groups to HIR, LIR execution, and generated Go projects. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 84ada03..6c98f26 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -48,6 +48,8 @@ Every top-level source expression becomes a synthetic declaration at its exact p Serialized `.fsir` files contain one closed resolved program plus explicit foreign linkage declarations. JSON format versions are internal and may be rejected rather than upgraded. +Artifact metadata records the actual compiler and language assembly versions, normalized paths and SHA-256 hashes for available file-backed sources, and a sorted unique list of foreign dependencies. Deserialization rejects missing metadata, malformed hashes, non-canonical dependency order, unknown formats or versions, and any LIR that fails post-load validation. + ## Representation policy - Primitive values, declared records, tuple shapes, and declared unions receive concrete representations where possible. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index ad942d7..6093084 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -53,3 +53,5 @@ Compiled maps use an ordered entry representation for string and integer keys. E ## Diagnostics Parse, type, and evaluation failures currently use source spans and exceptions. Compiler entry points translate expected failures into structured stage diagnostics while leaving existing embedding APIs behavior-compatible. + +Version-1 `.fsir` artifacts require compiler and language versions, source metadata, ordered dependencies, and a complete program. Unknown formats and versions are rejected before LIR construction; malformed metadata and invalid reconstructed LIR receive `FSA5xxx` diagnostics. diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index 4b71f79..d1b670c 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -12,6 +12,33 @@ module IrArtifact = [] let Version = 1 + let private assemblyVersion (assembly: Reflection.Assembly) = + match assembly.GetName().Version with null -> "0.0.0" | value -> value.ToString() + + let private normalizePath (path: string) = path.Replace('\\', '/') + + let private sourceMetadata (program: Lir.Program) = + let paths = + [ yield! program.Declarations |> List.choose (fun declaration -> declaration.Span.Start.File) + match program.Entry with Some entry -> yield! entry.Span.Start.File |> Option.toList | None -> () ] + |> List.distinct + |> List.sort + paths |> List.choose (fun path -> + if File.Exists path then + let hash = + File.ReadAllBytes path + |> Security.Cryptography.SHA256.HashData + |> Convert.ToHexString + |> _.ToLowerInvariant() + Some(normalizePath path, hash) + else None) + + let private dependencies (program: Lir.Program) = + program.Declarations + |> List.choose (fun declaration -> match declaration.Value.Node with Lir.HostExternal(name, _) -> Some name | _ -> None) + |> List.distinct + |> List.sort + let private writeOptionString (writer: Utf8JsonWriter) (name: string) (value: string option) = match value with | Some text -> writer.WriteString(name, text) @@ -212,8 +239,14 @@ module IrArtifact = use stream = new MemoryStream() use writer = new Utf8JsonWriter(stream, JsonWriterOptions(Indented = true)) writer.WriteStartObject(); writer.WriteString("format", Format); writer.WriteNumber("irVersion", Version) - writer.WriteString("compilerVersion", "0.0.0"); writer.WriteString("languageVersion", "0.0.0") - writer.WriteStartArray("sources"); writer.WriteEndArray(); writer.WriteStartArray("dependencies"); writer.WriteEndArray() + writer.WriteString("compilerVersion", assemblyVersion typeof.Assembly) + writer.WriteString("languageVersion", assemblyVersion typeof.Assembly) + writer.WriteStartArray("sources") + for path, hash in sourceMetadata program do + writer.WriteStartObject(); writer.WriteString("path", path); writer.WriteString("sha256", hash); writer.WriteEndObject() + writer.WriteEndArray(); writer.WriteStartArray("dependencies") + dependencies program |> List.iter writer.WriteStringValue + writer.WriteEndArray() writer.WriteStartObject("program"); writer.WriteStartArray("declarations") for declaration in program.Declarations do writer.WriteStartObject(); writer.WriteNumber("id", Lir.symbolNumber declaration.Id); writer.WriteString("name", declaration.Name) @@ -225,6 +258,9 @@ module IrArtifact = writer.WriteEndObject(); writer.WriteEndObject(); writer.Flush(); stream.ToArray() let private property (name: string) (element: JsonElement) = element.GetProperty(name) + let private hasProperty (name: string) (element: JsonElement) = + let mutable value = Unchecked.defaultof + element.TryGetProperty(name, &value) let private stringProperty (name: string) (element: JsonElement) = match (property name element).GetString() with null -> "" | value -> value let private readPosition (element: JsonElement) = @@ -361,15 +397,29 @@ module IrArtifact = let root = document.RootElement if stringProperty "format" root <> Format then Error [ { Code = "FSA5001"; Stage = Artifact; Message = "Unsupported IR artifact format."; Span = None } ] elif (property "irVersion" root).GetInt32() <> Version then Error [ { Code = "FSA5002"; Stage = Artifact; Message = "Incompatible IR artifact version."; Span = None } ] + elif [ "compilerVersion"; "languageVersion"; "sources"; "dependencies"; "program" ] |> List.exists (fun name -> not (hasProperty name root)) then + Error [ { Code = "FSA5004"; Stage = Artifact; Message = "IR artifact is missing required metadata."; Span = None } ] else - let programElement = property "program" root - let declarations = (property "declarations" programElement).EnumerateArray() |> Seq.map (fun (item: JsonElement) -> - ({ Id = Lir.SymbolId((property "id" item).GetInt32()); Name = stringProperty "name" item; Type = readType (property "type" item) - IsRecursive = (property "recursive" item).GetBoolean(); IsExported = (property "exported" item).GetBoolean() - Span = readSpan (property "span" item); Value = readExpression (property "value" item) }: Lir.Declaration)) |> Seq.toList - let entryElement = property "entry" programElement - let program : Lir.Program = - { Declarations = declarations - Entry = if entryElement.ValueKind = JsonValueKind.Null then None else Some(readExpression entryElement) } - LirValidation.validate program + let sourcesValid = + (property "sources" root).ValueKind = JsonValueKind.Array + && ((property "sources" root).EnumerateArray() |> Seq.forall (fun source -> + hasProperty "path" source && hasProperty "sha256" source + && (stringProperty "sha256" source).Length = 64)) + let dependencyValues = + if (property "dependencies" root).ValueKind = JsonValueKind.Array then + (property "dependencies" root).EnumerateArray() |> Seq.map _.GetString() |> Seq.choose Option.ofObj |> Seq.toList + else [] + if not sourcesValid || dependencyValues <> (dependencyValues |> List.distinct |> List.sort) then + Error [ { Code = "FSA5005"; Stage = Artifact; Message = "IR artifact metadata is invalid or non-canonical."; Span = None } ] + else + let programElement = property "program" root + let declarations = (property "declarations" programElement).EnumerateArray() |> Seq.map (fun (item: JsonElement) -> + ({ Id = Lir.SymbolId((property "id" item).GetInt32()); Name = stringProperty "name" item; Type = readType (property "type" item) + IsRecursive = (property "recursive" item).GetBoolean(); IsExported = (property "exported" item).GetBoolean() + Span = readSpan (property "span" item); Value = readExpression (property "value" item) }: Lir.Declaration)) |> Seq.toList + let entryElement = property "entry" programElement + let program : Lir.Program = + { Declarations = declarations + Entry = if entryElement.ValueKind = JsonValueKind.Null then None else Some(readExpression entryElement) } + LirValidation.validate program with ex -> Error [ { Code = "FSA5003"; Stage = Artifact; Message = ex.Message; Span = None } ] diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index 310ff81..564f4da 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -2,6 +2,8 @@ namespace FScript.Compiler.Tests open System open System.Text +open System.Text.Json +open System.Text.Json.Nodes open FScript.Compiler open FScript.Language open NUnit.Framework @@ -13,6 +15,11 @@ type IrArtifactTests() = | Ok value -> value | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + let parseObject (bytes: byte array) : JsonObject = + match JsonNode.Parse(Encoding.UTF8.GetString(bytes)) with + | null -> failwith "Expected JSON object" + | node -> node.AsObject() + [] member _.``IR artifact is deterministic and equivalent after round trip`` () = let compiled = compile "let add x y = x + y\n(Some (add 1 2), [1] @ [2])" @@ -37,6 +44,39 @@ type IrArtifactTests() = | Ok _ -> Assert.Fail("Expected corrupt-artifact diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSA5003")) + [] + member _.``IR artifact rejects missing and non-canonical metadata`` () = + let bytes = compile "1" |> _.Lir |> IrArtifact.serialize + let missing = parseObject bytes + missing.Remove("compilerVersion") |> ignore + match IrArtifact.deserialize (Encoding.UTF8.GetBytes(missing.ToJsonString())) with + | Ok _ -> Assert.Fail("Expected missing-metadata diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSA5004")) + + let invalid = parseObject bytes + let dependencies = JsonArray() + dependencies.Add("z") + dependencies.Add("a") + invalid["dependencies"] <- dependencies + match IrArtifact.deserialize (Encoding.UTF8.GetBytes(invalid.ToJsonString())) with + | Ok _ -> Assert.Fail("Expected non-canonical-metadata diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSA5005")) + + [] + member _.``IR artifact records normalized source paths and SHA256 hashes`` () = + let path = IO.Path.Combine(IO.Path.GetTempPath(), $"fscript-source-{Guid.NewGuid():N}.fss") + try + IO.File.WriteAllText(path, "1") + let input = { Program = Parser.parseProgramWithSourceName (Some path) "1"; Externs = [] } + let compiled = match Compiler.compile input with Ok value -> value | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + use document = JsonDocument.Parse(IrArtifact.serialize compiled.Lir) + let source = document.RootElement.GetProperty("sources")[0] + Assert.That(source.GetProperty("path").GetString(), Is.EqualTo(path.Replace('\\', '/'))) + let expected = Security.Cryptography.SHA256.HashData(IO.File.ReadAllBytes(path)) |> Convert.ToHexString |> _.ToLowerInvariant() + Assert.That(source.GetProperty("sha256").GetString(), Is.EqualTo(expected)) + finally + if IO.File.Exists path then IO.File.Delete path + [] member _.``IR artifact round trips lexical let and fail nodes`` () = for source in From a11e59e11dd55a551602976e42bd62c888cd66c0 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 17:53:56 +0200 Subject: [PATCH 026/133] feat(compiler): supply runtime execution context --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 ++ docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Backend.Go/GoBackend.fs | 18 ++++++++++++++++++ src/FScript.Compiler/Compiler.fs | 11 +++++++++-- src/FScript.Compiler/Hir.fs | 2 ++ src/FScript.Compiler/HirLowering.fs | 3 ++- src/FScript.Compiler/HirValidation.fs | 1 + src/FScript.Compiler/IrArtifact.fs | 2 ++ src/FScript.Compiler/Lir.fs | 2 ++ src/FScript.Interpreter.Lir/LirInterpreter.fs | 14 ++++++++++++-- .../FScript.Backend.Go.Tests/GoBackendTests.fs | 2 ++ .../FScript.Compiler.Tests/IrArtifactTests.fs | 3 ++- .../LirInterpreterTests.fs | 11 +++++++++++ 14 files changed, 67 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 42b69ff..39cd1ed 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added compiler runtime-context IR so LIR and generated Go programs receive `Env.ScriptName` and executable arguments without host linkage. - Added versioned `.fsir` compiler/language metadata, normalized source SHA-256 records, ordered dependencies, and strict metadata validation. - Exposed deterministic resolved frontend symbols, normalized declaration schemes, and instantiated lexical reference types without changing legacy inference APIs. - Added deterministic ordered-map and runtime type-reference patterns to HIR, artifacts, LIR execution, and generated Go. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 6c98f26..d3185e9 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -62,6 +62,8 @@ Artifact metadata records the actual compiler and language assembly versions, no Compiler bindings classify names as portable intrinsics, backend runtime primitives, target imports, or unsupported host externs. This metadata is distinct from executable .NET `ExternalFunction` implementations. +`Env` is a compiler runtime-context primitive, not a serialized host delegate. LIR interpretation materializes its named record from `InterpreterContext`; generated Go materializes the same fields from `os.Args`, including the executable base name and arguments after it. + The LIR interpreter rejects reachable target imports it cannot supply. The Go backend initially rejects unsupported host externs and tasks with backend diagnostics. Imports describe complete ordinary FScript declarations plus target linkage and do not create a second foreign type system. ## Previous experiment diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 6093084..3214324 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -47,6 +47,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Scalar conversions, string helpers, `List`, `Option`, and `Map` functions are protected builtin symbols with explicit polymorphic schemes. They are initial compiler/runtime intrinsics even where a future implementation can compile their FScript source. - The LIR interpreter and Go runtime execute the complete pure builtin catalog, including higher-order list, option, and ordered-map functions. Callback invocation remains curried and follows source collection order. - `Console.writeLine`, `Console.readLine`, and the CLI `Env` value are candidate portable runtime primitives. Filesystem, regex, hashing, GUID, JSON, and XML functions remain host externs initially. +- Compiled `Env` is a runtime-context intrinsic. The LIR interpreter uses its explicit script name and arguments; generated Go uses the executable base name and `os.Args[1:]`. - `Task.spawn` and `Task.await` are host externs with tracked-task semantics, background error propagation, and an error for unawaited tasks. HIR may represent them before LIR interpreters or backends support them. - `[]` is declaration metadata used by embedding hosts and does not change inference. Backend `Import` metadata is separate and will not weaken the declared FScript type. diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index 5fdd1db..47cc656 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -21,6 +21,7 @@ module GoBackend = | TList _ -> "[]any" | TOption _ -> "fsruntime.Option" | TRecord _ -> "map[string]any" + | TNamed "Environment" -> "map[string]any" | TUnion _ | TNamed _ -> "fsruntime.Union" | TMap _ -> "fsruntime.OrderedMap" | value -> invalidArg "value" $"Unsupported Go representation for {Types.typeToString value}." @@ -59,6 +60,7 @@ module GoBackend = | TList _ | TTuple _ -> $"fsruntime.List({code})" | TRecord _ -> $"fsruntime.Record({code})" | TOption _ -> $"fsruntime.AsOption({code})" + | TNamed "Environment" -> $"fsruntime.Record({code})" | TUnion _ | TNamed _ -> $"fsruntime.AsUnion({code})" | TMap _ -> $"fsruntime.AsMap({code})" | TFun _ -> $"fsruntime.AsClosure({code})" @@ -325,6 +327,7 @@ module GoBackend = $"fsruntime.BuildMap([]fsruntime.MapPart{{{joined}}})") | Lir.Intrinsic name -> Ok(emitIntrinsic name expression.Type) | Lir.HostExternal(name, _) -> Ok(emitHostExternal name expression.Type) + | Lir.RuntimeContext -> Ok "fsruntime.Environment()" | Lir.ForEach(id, _, itemType, source, body) -> let name = identifier "forValue" id let scoped = environment |> Map.add id name @@ -413,6 +416,20 @@ func matchRuntimePattern(value any, pattern Pattern) (map[int]any, bool) { func PatternMatches(value any, pattern Pattern) bool { _, matched := matchRuntimePattern(value, pattern); return matched } func PatternBinding(value any, pattern Pattern, symbol int) any { bindings, matched := matchRuntimePattern(value, pattern); if !matched { panic("Pattern did not match") }; return bindings[symbol] } func TypeMatches(value any, typeName string) bool { switch typeName { case "unit": _, ok := value.(struct{}); return ok; case "int": _, ok := value.(int64); return ok; case "float": _, ok := value.(float64); return ok; case "bool": _, ok := value.(bool); return ok; case "string": _, ok := value.(string); return ok }; if strings.HasSuffix(typeName, " list") { _, ok := value.([]any); return ok }; if strings.HasSuffix(typeName, " option") { _, ok := value.(Option); return ok }; if strings.HasPrefix(typeName, "{") { _, ok := value.(map[string]any); return ok }; if item, ok := value.(Union); ok { return item.TypeName == typeName }; _, record := value.(map[string]any); return record } +""" + + let private environmentRuntime = """package fsruntime + +import ( + "os" + "path/filepath" +) + +func Environment() map[string]any { + arguments := make([]any, len(os.Args)-1) + for index, argument := range os.Args[1:] { arguments[index] = argument } + return map[string]any{"ScriptName": Some(filepath.Base(os.Args[0])), "Arguments": arguments} +} """ let generate options (program: Lir.Program) : Result = @@ -450,6 +467,7 @@ func TypeMatches(value any, typeName string) bool { switch typeName { case "unit { RelativePath = "internal/fscriptprogram/program.go"; Content = programSource } { RelativePath = "internal/fsruntime/closures.go"; Content = closureRuntime } { RelativePath = "internal/fsruntime/patterns.go"; Content = patternRuntime } + { RelativePath = "internal/fsruntime/environment.go"; Content = environmentRuntime } { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"sort\"\n\t\"strconv\"\n\t\"strings\"\n)\n\ntype Option struct { HasValue bool; Value any }\ntype Union struct { TypeName string; CaseName string; HasValue bool; Value any }\ntype MapEntry struct { Key any; Value any }\ntype OrderedMap struct { Entries []MapEntry }\ntype MapPart struct { IsSpread bool; Key any; Value any; Map OrderedMap }\n\nfunc Unit(value any) struct{} { return struct{}{} }\nfunc Int(value any) int64 { return value.(int64) }\nfunc Float(value any) float64 { return value.(float64) }\nfunc Bool(value any) bool { return value.(bool) }\nfunc String(value any) string { return value.(string) }\nfunc List(value any) []any { return value.([]any) }\nfunc Record(value any) map[string]any { return value.(map[string]any) }\nfunc AsOption(value any) Option { return value.(Option) }\nfunc AsUnion(value any) Union { return value.(Union) }\nfunc AsMap(value any) OrderedMap { return value.(OrderedMap) }\nfunc Some(value any) Option { return Option{HasValue: true, Value: value} }\nfunc None() Option { return Option{} }\nfunc IsSome(value any) bool { return AsOption(value).HasValue }\nfunc OptionValue(value any) any { return AsOption(value).Value }\nfunc MakeUnion(typeName string, caseName string) Union { return Union{TypeName: typeName, CaseName: caseName} }\nfunc MakeUnionValue(typeName string, caseName string, value any) Union { return Union{TypeName: typeName, CaseName: caseName, HasValue: true, Value: value} }\nfunc IsUnion(value any, typeName string, caseName string) bool { item := AsUnion(value); return item.TypeName == typeName && item.CaseName == caseName }\nfunc UnionHasValue(value any) bool { return AsUnion(value).HasValue }\nfunc UnionValue(value any) any { return AsUnion(value).Value }\nfunc HasField(value any, name string) bool { _, ok := Record(value)[name]; return ok }\nfunc Equal(left any, right any) bool { return reflect.DeepEqual(left, right) }\nfunc Cons(head any, tail []any) []any { return append([]any{head}, tail...) }\nfunc Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) }\nfunc Tail(value any) []any { return append([]any{}, List(value)[1:]...) }\nfunc Range(first int64, last int64) []any { step := int64(1); if first > last { step = -1 }; values := []any{}; for current := first; ; current += step { values = append(values, current); if current == last { break } }; return values }\nfunc UpdateRecord(value any, updates map[string]any) map[string]any { result := map[string]any{}; for key, item := range Record(value) { result[key] = item }; for key, item := range updates { result[key] = item }; return result }\nfunc Pair(key any, value any) MapPart { return MapPart{Key: key, Value: value} }\nfunc SpreadMap(value any) MapPart { return MapPart{IsSpread: true, Map: AsMap(value)} }\nfunc mapSet(entries []MapEntry, key any, value any, replace bool) []MapEntry { for index, entry := range entries { if Equal(entry.Key, key) { if replace { entries[index].Value = value }; return entries } }; return append(entries, MapEntry{Key: key, Value: value}) }\nfunc BuildMap(parts []MapPart) OrderedMap { entries := []MapEntry{}; for _, part := range parts { if part.IsSpread { for _, entry := range part.Map.Entries { entries = mapSet(entries, entry.Key, entry.Value, false) } } else { entries = mapSet(entries, part.Key, part.Value, true) } }; sort.SliceStable(entries, func(left int, right int) bool { return Format(entries[left].Key) < Format(entries[right].Key) }); return OrderedMap{Entries: entries} }\nfunc Index(value any, index any) Option { switch target := value.(type) { case []any: position := Int(index); if position < 0 || position >= int64(len(target)) { return None() }; return Some(target[position]); case OrderedMap: for _, entry := range target.Entries { if Equal(entry.Key, index) { return Some(entry.Value) } }; return None(); default: panic(\"Index access requires a list or map\") } }\nfunc Intrinsic(name string, args []any) any { switch name { case \"ignore\": return struct{}{}; case \"List.empty\": return []any{}; case \"Map.empty\": return OrderedMap{}; case \"Int.tryParse\": value, error := strconv.ParseInt(String(args[0]), 10, 64); if error != nil { return None() }; return Some(value); case \"Float.tryParse\": value, error := strconv.ParseFloat(String(args[0]), 64); if error != nil { return None() }; return Some(value); case \"Bool.tryParse\": value, error := strconv.ParseBool(String(args[0])); if error != nil { return None() }; return Some(value); case \"Int.toString\", \"Float.toString\", \"Bool.toString\": return Format(args[0]); case \"String.replace\": return strings.ReplaceAll(String(args[2]), String(args[0]), String(args[1])); case \"String.indexOf\": index := strings.Index(String(args[1]), String(args[0])); if index < 0 { return None() }; return Some(int64(index)); case \"String.toLower\": return strings.ToLower(String(args[0])); case \"String.toUpper\": return strings.ToUpper(String(args[0])); case \"String.substring\": start, length, value := Int(args[0]), Int(args[1]), String(args[2]); if start < 0 || length < 0 || start + length > int64(len(value)) { return None() }; return Some(value[start:start+length]); case \"String.concat\": values := List(args[1]); parts := make([]string, len(values)); for index, value := range values { parts[index] = String(value) }; return strings.Join(parts, String(args[0])); case \"String.split\": values := strings.Split(String(args[1]), String(args[0])); result := make([]any, len(values)); for index, value := range values { result[index] = value }; return result; case \"String.endsWith\": return strings.HasSuffix(String(args[1]), String(args[0])); case \"List.length\": return int64(len(List(args[0]))); case \"List.rev\": values := List(args[0]); result := make([]any, len(values)); for index, value := range values { result[len(values)-index-1] = value }; return result; case \"List.tryHead\": values := List(args[0]); if len(values) == 0 { return None() }; return Some(values[0]); case \"List.tail\": return Tail(args[0]); case \"List.append\": return Append(List(args[0]), List(args[1])); case \"List.contains\": for _, value := range List(args[1]) { if Equal(args[0], value) { return true } }; return false; case \"Option.isNone\": return !IsSome(args[0]); case \"Option.isSome\": return IsSome(args[0]); case \"Option.defaultValue\": option := AsOption(args[1]); if option.HasValue { return option.Value }; return args[0]; case \"Map.count\": return int64(len(AsMap(args[0]).Entries)); case \"Map.containsKey\": return IsSome(Index(args[1], args[0])); case \"Map.tryGet\": return Index(args[1], args[0]); default: panic(\"Unsupported FScript intrinsic: \" + name) } }\nfunc Format(value any) string { switch item := value.(type) { case bool: if item { return \"true\" }; return \"false\"; case Option: if !item.HasValue { return \"None\" }; return \"Some \" + Format(item.Value); case Union: if !item.HasValue { return item.CaseName }; return item.CaseName + \" \" + Format(item.Value); case OrderedMap: parts := make([]string, len(item.Entries)); for index, entry := range item.Entries { parts[index] = \"[\" + Format(entry.Key) + \"] = \" + Format(entry.Value) }; return \"{ \" + strings.Join(parts, \"; \") + \" }\"; case []any: parts := make([]string, len(item)); for index, value := range item { parts[index] = Format(value) }; return \"[\" + strings.Join(parts, \";\") + \"]\"; default: return fmt.Sprint(value) } }\nfunc Interpolate(values []any) string { var result strings.Builder; for _, value := range values { result.WriteString(Format(value)) }; return result.String() }\nfunc Fail(value any) { panic(fmt.Sprint(value)) }\n" } ] with ex -> Error [ { Code = "FSB4002"; Stage = GoGeneration; Message = ex.Message; Span = None } ] diff --git a/src/FScript.Compiler/Compiler.fs b/src/FScript.Compiler/Compiler.fs index 1c474c5..a09e107 100644 --- a/src/FScript.Compiler/Compiler.fs +++ b/src/FScript.Compiler/Compiler.fs @@ -11,11 +11,18 @@ type CompiledProgram = Lir: Lir.Program } module Compiler = + let private runtimeContext : ExternalFunction = + { Name = "Env" + Scheme = Forall([], TNamed "Environment") + Arity = 0 + Impl = fun _ _ -> VUnit } + let compile input = try + let externs = runtimeContext :: (input.Externs |> List.filter (fun external -> external.Name <> runtimeContext.Name)) input.Program - |> TypeInfer.inferResolvedProgramWithExterns input.Externs - |> HirLowering.lower input.Externs + |> TypeInfer.inferResolvedProgramWithExterns externs + |> HirLowering.lower externs |> Result.bind HirValidation.validate |> Result.bind (fun hir -> hir |> LirLowering.lower |> LirValidation.validate diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs index 5c982ed..48dd1dc 100644 --- a/src/FScript.Compiler/Hir.fs +++ b/src/FScript.Compiler/Hir.fs @@ -62,6 +62,7 @@ module Hir = | Map of MapEntry list | Intrinsic of string | HostExternal of string * int + | RuntimeContext | ForEach of SymbolId * string * Type * Expression * Expression | TypeToken of Type @@ -188,6 +189,7 @@ module Hir = entries |> List.iter (function KeyValue(key, value) -> expression (indent + 2) key; expression (indent + 2) value | Spread value -> expression (indent + 2) value) | Intrinsic name -> lines.Add(typed $"intrinsic {name}") | HostExternal(name, arity) -> lines.Add(typed $"host-extern {name}/{arity}") + | RuntimeContext -> lines.Add(typed "runtime-context") | ForEach(id, name, _, source, body) -> lines.Add(typed (sprintf "for %%%d %s" (symbolNumber id) name)); expression (indent + 2) source; expression (indent + 2) body | TypeToken tokenType -> lines.Add(typed $"typeof {formatType tokenType}") diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index a76fa71..a55b582 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -366,7 +366,8 @@ module HirLowering = externs |> List.sortBy _.Name |> List.map (fun external -> let (Forall(_, valueType)) = external.Scheme let span = Span.mk (Span.pos 0 0) (Span.pos 0 0) - ({ Id = externSymbols.[external.Name]; Name = external.Name; Scheme = external.Scheme; IsRecursive = false; IsExported = false; Span = span; Value = { Type = valueType; Span = span; Node = Hir.HostExternal(external.Name, external.Arity) } }: Hir.Declaration)) + let node = if external.Name = "Env" then Hir.RuntimeContext else Hir.HostExternal(external.Name, external.Arity) + ({ Id = externSymbols.[external.Name]; Name = external.Name; Scheme = external.Scheme; IsRecursive = false; IsExported = false; Span = span; Value = { Type = valueType; Span = span; Node = node } }: Hir.Declaration)) builtinDeclarations @ externalDeclarations @ (constructorSymbols |> List.map (fun (typeName, caseName, id) -> let definition, unionType = unionDefinitions |> List.find (fun (definition, _) -> definition.Name = typeName) let payloadType = definition.Cases |> List.find (fst >> (=) caseName) |> snd |> Option.map typeFromRef diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index d69d3c3..f64a46c 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -60,6 +60,7 @@ module HirValidation = entries |> List.map (function Hir.KeyValue(key, value) -> Set.union (references key) (references value) | Hir.Spread value -> references value) |> Set.unionMany | Hir.Intrinsic _ -> Set.empty | Hir.HostExternal _ -> Set.empty + | Hir.RuntimeContext -> Set.empty | Hir.ForEach(id, _, _, source, body) -> Set.union (references source) (references body |> Set.remove id) | Hir.TypeToken _ -> Set.empty let declared = declarationIds |> Set.ofList diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index d1b670c..c5a8e71 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -228,6 +228,7 @@ module IrArtifact = writer.WriteEndArray() | Lir.Intrinsic name -> writer.WriteString("kind", "intrinsic"); writer.WriteString("name", name) | Lir.HostExternal(name, arity) -> writer.WriteString("kind", "host-external"); writer.WriteString("name", name); writer.WriteNumber("arity", arity) + | Lir.RuntimeContext -> writer.WriteString("kind", "runtime-context") | Lir.ForEach(id, name, itemType, source, body) -> writer.WriteString("kind", "for-each"); writer.WriteNumber("symbol", Lir.symbolNumber id); writer.WriteString("name", name) writer.WritePropertyName("itemType"); writeType writer itemType; writer.WritePropertyName("source"); writeExpression writer source @@ -385,6 +386,7 @@ module IrArtifact = | kind -> invalidOp $"Unknown map entry '{kind}'.") |> Seq.toList) | "intrinsic" -> Lir.Intrinsic(stringProperty "name" element) | "host-external" -> Lir.HostExternal(stringProperty "name" element, (property "arity" element).GetInt32()) + | "runtime-context" -> Lir.RuntimeContext | "for-each" -> Lir.ForEach(Lir.SymbolId((property "symbol" element).GetInt32()), stringProperty "name" element, readType (property "itemType" element), readExpression (property "source" element), readExpression (property "body" element)) | "type-token" -> Lir.TypeToken(readType (property "tokenType" element)) diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index 0604741..a59a937 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -50,6 +50,7 @@ module Lir = | Map of MapEntry list | Intrinsic of string | HostExternal of string * int + | RuntimeContext | ForEach of SymbolId * string * Type * Expression * Expression | TypeToken of Type @@ -171,6 +172,7 @@ module LirLowering = | Hir.Map entries -> Lir.Map(entries |> List.map (function Hir.KeyValue(key, item) -> Lir.KeyValue(expression key, expression item) | Hir.Spread item -> Lir.Spread(expression item))) | Hir.Intrinsic name -> Lir.Intrinsic name | Hir.HostExternal(name, arity) -> Lir.HostExternal(name, arity) + | Hir.RuntimeContext -> Lir.RuntimeContext | Hir.ForEach(id, name, itemType, source, body) -> Lir.ForEach(symbol id, name, itemType, expression source, expression body) | Hir.TypeToken tokenType -> Lir.TypeToken tokenType { Type = value.Type; Span = value.Span; Node = node } diff --git a/src/FScript.Interpreter.Lir/LirInterpreter.fs b/src/FScript.Interpreter.Lir/LirInterpreter.fs index 9dd482f..1730344 100644 --- a/src/FScript.Interpreter.Lir/LirInterpreter.fs +++ b/src/FScript.Interpreter.Lir/LirInterpreter.fs @@ -21,6 +21,7 @@ type LirValue = | VIntrinsic of string * LirValue list | VExternal of string * int * LirValue list | VTypeToken of Type + | VRuntimeContext | VClosure of Lir.SymbolId * string * Lir.Expression * Map ref type ExecutionResult = { Value: LirValue } @@ -62,6 +63,7 @@ module LirInterpreter = | VIntrinsic _ -> "" | VExternal _ -> "" | VTypeToken value -> $"" + | VRuntimeContext -> "" let private fail code span message = Error [ { Code = code; Stage = Interpreter; Message = message; Span = Some span } ] @@ -394,6 +396,7 @@ module LirInterpreter = if intrinsicArity name = 0 then invokeIntrinsic (applyValue expression.Span) expression.Span name [] else Ok(VIntrinsic(name, [])) | Lir.HostExternal(name, arity) -> Ok(VExternal(name, arity, [])) + | Lir.RuntimeContext -> Ok VRuntimeContext | Lir.ForEach(id, _, _, source, body) -> match evaluate environment source with | Ok(VList values) -> @@ -425,7 +428,11 @@ module LirInterpreter = results |> List.fold (fun state result -> match state, result with Ok values, Ok value -> Ok(values @ [ value ]) | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e) (Ok []) - let execute (_context: InterpreterContext) (program: Lir.Program) = + let execute (context: InterpreterContext) (program: Lir.Program) = + let runtimeContext = + VRecord(Map.ofList + [ "ScriptName", VOption(Option.map VString context.ScriptName) + "Arguments", VList(List.map VString context.Arguments) ]) let environment = ref Map.empty let mutable failure = None let rec initialize (declarations: Lir.Declaration list) = @@ -434,7 +441,10 @@ module LirInterpreter = | declaration :: rest when not declaration.IsRecursive -> match evaluate environment.Value declaration.Value with | Error errors -> failure <- Some errors - | Ok value -> environment.Value <- environment.Value |> Map.add declaration.Id value; initialize rest + | Ok value -> + let initialized = match value with VRuntimeContext -> runtimeContext | _ -> value + environment.Value <- environment.Value |> Map.add declaration.Id initialized + initialize rest | _ -> let group = declarations |> List.takeWhile _.IsRecursive let remainder = declarations |> List.skip group.Length diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index 3de1624..e025cc1 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -50,6 +50,8 @@ type GoBackendTests() = [] [ value\n | {} -> 0", "1")>] [ true\n | _ -> false", "true")>] + [] + [] member _.``Generated projects run with Go`` (source: string, expected: string) = let input = { Program = Parser.parseProgram source; Externs = [] } let compiled = diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index 564f4da..6de98ed 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -92,7 +92,8 @@ type IrArtifactTests() = "type Package = { Name: string }\ntypeof Package" "(let rec even n = if n = 0 then true else odd (n - 1)\nand odd n = if n = 0 then false else even (n - 1)\neven 10\n)" "match { [\"a\"] = 1 } with\n | { [key] = value; ..tail } -> value\n | {} -> 0" - "match {| Name = \"Ada\" |} with\n | {| Name: string |} -> true\n | _ -> false" ] do + "match {| Name = \"Ada\" |} with\n | {| Name: string |} -> true\n | _ -> false" + "List.length Env.Arguments" ] do let bytes = compile source |> _.Lir |> IrArtifact.serialize match IrArtifact.deserialize bytes with | Ok _ -> () diff --git a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs index c819656..7134695 100644 --- a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs +++ b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs @@ -208,3 +208,14 @@ type LirInterpreterTests() = match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok _ -> Assert.Pass() | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``Supplies Env from the LIR interpreter context`` () = + let source = "(Env.ScriptName, Env.Arguments)" + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = Some "sample.fss"; Arguments = [ "one"; "two" ] } compiled.Lir with + | Ok result -> + Assert.That(result.Value, Is.EqualTo(LirValue.VTuple [ LirValue.VOption(Some(LirValue.VString "sample.fss")); LirValue.VList [ LirValue.VString "one"; LirValue.VString "two" ] ])) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) From a469559eed15ed5de3a5fe3e8a3d33a34f817b36 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 18:01:28 +0200 Subject: [PATCH 027/133] feat(go): emit initial Go import bindings --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 + docs/specs/syntax-and-indentation.md | 3 ++ src/FScript.Backend.Go/GoBackend.fs | 45 ++++++++++++++-- .../LanguageServer/LspSymbols.fs | 4 ++ src/FScript.Compiler/Hir.fs | 2 + src/FScript.Compiler/HirLowering.fs | 10 ++++ src/FScript.Compiler/HirValidation.fs | 1 + src/FScript.Compiler/IrArtifact.fs | 8 ++- src/FScript.Compiler/Lir.fs | 2 + src/FScript.Interpreter.Lir/LirInterpreter.fs | 9 ++++ src/FScript.Language/Ast.fs | 11 ++++ src/FScript.Language/Eval.fs | 8 +++ src/FScript.Language/IncludeResolver.fs | 2 + src/FScript.Language/Interactive.fs | 1 + src/FScript.Language/Parser.fs | 53 ++++++++++++++++--- src/FScript.Language/TypeInfer.fs | 11 ++++ .../GoBackendTests.fs | 1 + .../FScript.Compiler.Tests/IrArtifactTests.fs | 3 +- .../LirInterpreterTests.fs | 16 ++++++ tests/FScript.Language.Tests/ParserTests.fs | 13 +++++ 21 files changed, 192 insertions(+), 14 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 39cd1ed..0c8273e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added bodyless `[]` declarations with typed HIR/LIR linkage and primitive Go package bindings. - Added compiler runtime-context IR so LIR and generated Go programs receive `Env.ScriptName` and executable arguments without host linkage. - Added versioned `.fsir` compiler/language metadata, normalized source SHA-256 records, ordered dependencies, and strict metadata validation. - Exposed deterministic resolved frontend symbols, normalized declaration schemes, and instantiated lexical reference types without changing legacy inference APIs. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index d3185e9..a880d50 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -66,6 +66,8 @@ Compiler bindings classify names as portable intrinsics, backend runtime primiti The LIR interpreter rejects reachable target imports it cannot supply. The Go backend initially rejects unsupported host externs and tasks with backend diagnostics. Imports describe complete ordinary FScript declarations plus target linkage and do not create a second foreign type system. +Initial Go imports use `[] let name : complete -> signature` with no body. The frontend requires a function type and records linkage separately from its FScript signature. Go emission uses deterministic package aliases and currently accepts only primitive/unit arguments and results; other representations receive `FSB4003`. The LIR interpreter keeps unused imports inert and reports `FSI3023` only when one is fully applied. + ## Previous experiment The `feature/in-memory-execution` branch is useful only as evidence about closures, invocation, and host integration. Its dynamic-method/in-memory assembly backend is not an IR layer and is neither copied nor referenced by the new compiler architecture. diff --git a/docs/specs/syntax-and-indentation.md b/docs/specs/syntax-and-indentation.md index e422e13..fcdca71 100644 --- a/docs/specs/syntax-and-indentation.md +++ b/docs/specs/syntax-and-indentation.md @@ -10,10 +10,13 @@ This document describes the concrete syntax accepted by the interpreter and the - `type` / `type rec` declarations - `let` / `let rec` bindings - `[] let` / `[] let rec` bindings + - bodyless `[] let name : complete -> signature` declarations - expressions - Parsing is layout-aware: the lexer emits `Indent`/`Dedent` tokens from leading whitespace. - Comments use `//` line-comment syntax. +Backend `Import` requires exactly a target module path and member name, a complete function signature, and no body. It cannot be combined with `export` and is not allowed in expression blocks. The legacy evaluator reports an explicit error if an imported function is called; compiled consumers use the retained target linkage. + ## Core expression forms - Literals: `int`, `float`, `bool`, `string`, `()` - Variables and function application by whitespace: diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index 47cc656..50efd33 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -8,6 +8,10 @@ type GoOptions = { ModuleName: string } type GeneratedFile = { RelativePath: string; Content: string } module GoBackend = + let private importAlias (modulePath: string) = + let hash = Security.Cryptography.SHA256.HashData(Encoding.UTF8.GetBytes(modulePath)) |> Convert.ToHexString + "goimport" + hash.Substring(0, 8).ToLowerInvariant() + let rec private goType = function | TUnit -> "struct{}" | TInt -> "int64" @@ -189,6 +193,21 @@ module GoBackend = cast resultType $"fsruntime.Intrinsic(\"__host:\" + {quoted}, []any{{{values}}})" build 0 [] externalType + let private emitTargetImport modulePath memberName importType = + let alias = importAlias modulePath + let rec build index arguments = function + | TFun(argumentType, resultType) -> + let raw = $"targetArg{index}" + let argument = $"targetValue{index}" + let body = build (index + 1) (arguments @ [ argument ]) resultType + $"fsruntime.Closure{{Call: func({raw} any) any {{ {argument} := {cast argumentType raw}; return {body} }}}}" + | resultType -> + let joined = String.concat ", " arguments + match resultType with + | TUnit -> $"func() any {{ {alias}.{memberName}({joined}); return struct{{}}{{}} }}()" + | _ -> $"{alias}.{memberName}({joined})" + build 0 [] importType + let rec private emitExpression environment (expression: Lir.Expression) = match expression.Node with | Lir.Constant value -> Ok(constant value) @@ -328,6 +347,7 @@ module GoBackend = | Lir.Intrinsic name -> Ok(emitIntrinsic name expression.Type) | Lir.HostExternal(name, _) -> Ok(emitHostExternal name expression.Type) | Lir.RuntimeContext -> Ok "fsruntime.Environment()" + | Lir.TargetImport(modulePath, memberName) -> Ok(emitTargetImport modulePath memberName expression.Type) | Lir.ForEach(id, _, itemType, source, body) -> let name = identifier "forValue" id let scoped = environment |> Map.add id name @@ -434,6 +454,15 @@ func Environment() map[string]any { let generate options (program: Lir.Program) : Result = try + let targetImports = + program.Declarations + |> List.choose (fun declaration -> match declaration.Value.Node with Lir.TargetImport(modulePath, memberName) -> Some(modulePath, memberName, declaration.Type, declaration.Span) | _ -> None) + |> List.sortBy (fun (modulePath, memberName, _, _) -> modulePath, memberName) + let rec directlyRepresentable = function + | TUnit | TInt | TFloat | TBool | TString -> true + | TFun(argument, result) -> directlyRepresentable argument && directlyRepresentable result + | _ -> false + let unsupportedImport = targetImports |> List.tryFind (fun (_, _, importType, _) -> not (directlyRepresentable importType)) let names = program.Declarations |> List.fold (fun env declaration -> env |> Map.add declaration.Id (identifier "value" declaration.Id)) Map.empty @@ -449,14 +478,20 @@ func Environment() map[string]any { match program.Entry with | None -> Ok "struct{}{}" | Some value -> emitExpression names value - match error, entry with - | Some(message, span), _ -> + match unsupportedImport, error, entry with + | Some(modulePath, memberName, _, span), _, _ -> + Error [ { Code = "FSB4003"; Stage = GoGeneration; Message = $"Go import '{modulePath}.{memberName}' uses a type that is not directly representable."; Span = Some span } ] + | None, Some(message, span), _ -> Error [ { Code = "FSB4001"; Stage = GoGeneration; Message = message; Span = Some span } ] - | None, Error message -> + | None, None, Error message -> Error [ { Code = "FSB4001"; Stage = GoGeneration; Message = message; Span = None } ] - | None, Ok entryCode -> + | None, None, Ok entryCode -> let moduleName = if String.IsNullOrWhiteSpace options.ModuleName then "fscript.generated/program" else options.ModuleName - let programSource = $"package fscriptprogram\n\nimport \"{moduleName}/internal/fsruntime\"\n\n{declarations}\nfunc init() {{\n{initializers}}}\n\nfunc Run() any {{ return {entryCode} }}\n" + let importLines = + ($"\t\"{moduleName}/internal/fsruntime\"") :: + (targetImports |> List.map (fun (modulePath, _, _, _) -> modulePath) |> List.distinct |> List.map (fun path -> $"\t{importAlias path} {System.Text.Json.JsonSerializer.Serialize(path)}")) + |> String.concat "\n" + let programSource = $"package fscriptprogram\n\nimport (\n{importLines}\n)\n\n{declarations}\nfunc init() {{\n{initializers}}}\n\nfunc Run() any {{ return {entryCode} }}\n" let printExpression = match program.Entry with | Some { Type = TUnit } -> "_ = fsruntime.Format(fscriptprogram.Run()); fmt.Println(\"()\")" diff --git a/src/FScript.CSharpInterop/LanguageServer/LspSymbols.fs b/src/FScript.CSharpInterop/LanguageServer/LspSymbols.fs index fbc5d0f..4eb2cd8 100644 --- a/src/FScript.CSharpInterop/LanguageServer/LspSymbols.fs +++ b/src/FScript.CSharpInterop/LanguageServer/LspSymbols.fs @@ -872,6 +872,7 @@ let queryValues : string -> string -> string list option bindings |> List.tryPick (fun (name, _, bindingType, _) -> if name = targetName then Some bindingType else None) + | TypeInfer.TSBackendImport(name, importedType, _, _) when name = targetName -> Some importedType | _ -> None) let keyOf (entry: TypeInfer.LocalVariableTypeInfo) = @@ -884,6 +885,7 @@ let queryValues : string -> string -> string list option | SLet(name, _, _, _, _, _, _) -> [ name ] | SLetPattern(pattern, _, _, _) -> collectPatternNames pattern | SLetRecGroup(bindings, _, _) -> bindings |> List.map (fun (name, _, _, _, _) -> name) + | SBackendImport(name, _, _, _) -> [ name ] | SType _ | SExpr _ | SImport _ -> [] @@ -1193,6 +1195,8 @@ let queryValues : string -> string -> string list option let scopedWithArgs = pushBindings scoped argBindings resolveExpr scopedWithArgs valueExpr)) resolveStmts scopesWithNames rest (acc @ inBindings) + | SBackendImport(name, _, _, span) -> + resolveStmts (pushBindings scopes [ name, span ]) rest acc | SExpr expr -> let inExpr = resolveExpr scopes expr resolveStmts scopes rest (acc @ inExpr) diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs index 48dd1dc..85dc277 100644 --- a/src/FScript.Compiler/Hir.fs +++ b/src/FScript.Compiler/Hir.fs @@ -63,6 +63,7 @@ module Hir = | Intrinsic of string | HostExternal of string * int | RuntimeContext + | TargetImport of string * string | ForEach of SymbolId * string * Type * Expression * Expression | TypeToken of Type @@ -190,6 +191,7 @@ module Hir = | Intrinsic name -> lines.Add(typed $"intrinsic {name}") | HostExternal(name, arity) -> lines.Add(typed $"host-extern {name}/{arity}") | RuntimeContext -> lines.Add(typed "runtime-context") + | TargetImport(modulePath, memberName) -> lines.Add(typed $"target-import {modulePath}.{memberName}") | ForEach(id, name, _, source, body) -> lines.Add(typed (sprintf "for %%%d %s" (symbolNumber id) name)); expression (indent + 2) source; expression (indent + 2) body | TypeToken tokenType -> lines.Add(typed $"typeof {formatType tokenType}") diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index a55b582..e420ff8 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -350,6 +350,7 @@ module HirLowering = | TypeInfer.TSLet(name, _, _, _, _, _) -> [ name ] | TypeInfer.TSLetPattern(_, _, bindings, _, _) -> bindings |> Map.keys |> Seq.toList | TypeInfer.TSLetRecGroup(bindings, _, _) -> bindings |> List.map (fun (name, _, _, _) -> name) + | TypeInfer.TSBackendImport(name, _, _, _) -> [ name ] | _ -> []) |> List.distinct |> List.fold (fun env name -> env |> Map.add name (newSymbol state)) (Map.fold (fun env name id -> Map.add name id env) builtins externSymbols) @@ -452,6 +453,15 @@ module HirLowering = Span = span Value = value }: Hir.Declaration) ] | Error diagnostics -> errors <- errors @ diagnostics + | TypeInfer.TSBackendImport(name, importedType, import, span) -> + declarations <- declarations @ + [ ({ Id = Map.find name topLevel + Name = name + Scheme = Forall([], importedType) + IsRecursive = false + IsExported = false + Span = span + Value = { Type = importedType; Span = span; Node = Hir.TargetImport(import.ModulePath, import.MemberName) } }: Hir.Declaration) ] | TypeInfer.TSType _ -> () if errors.IsEmpty then Ok { Declarations = declarations; Entry = entry } else Error errors diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index f64a46c..3c0173d 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -61,6 +61,7 @@ module HirValidation = | Hir.Intrinsic _ -> Set.empty | Hir.HostExternal _ -> Set.empty | Hir.RuntimeContext -> Set.empty + | Hir.TargetImport _ -> Set.empty | Hir.ForEach(id, _, _, source, body) -> Set.union (references source) (references body |> Set.remove id) | Hir.TypeToken _ -> Set.empty let declared = declarationIds |> Set.ofList diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index c5a8e71..aef3d26 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -35,7 +35,11 @@ module IrArtifact = let private dependencies (program: Lir.Program) = program.Declarations - |> List.choose (fun declaration -> match declaration.Value.Node with Lir.HostExternal(name, _) -> Some name | _ -> None) + |> List.choose (fun declaration -> + match declaration.Value.Node with + | Lir.HostExternal(name, _) -> Some $"host:{name}" + | Lir.TargetImport(modulePath, memberName) -> Some $"target:{modulePath}:{memberName}" + | _ -> None) |> List.distinct |> List.sort @@ -229,6 +233,7 @@ module IrArtifact = | Lir.Intrinsic name -> writer.WriteString("kind", "intrinsic"); writer.WriteString("name", name) | Lir.HostExternal(name, arity) -> writer.WriteString("kind", "host-external"); writer.WriteString("name", name); writer.WriteNumber("arity", arity) | Lir.RuntimeContext -> writer.WriteString("kind", "runtime-context") + | Lir.TargetImport(modulePath, memberName) -> writer.WriteString("kind", "target-import"); writer.WriteString("modulePath", modulePath); writer.WriteString("memberName", memberName) | Lir.ForEach(id, name, itemType, source, body) -> writer.WriteString("kind", "for-each"); writer.WriteNumber("symbol", Lir.symbolNumber id); writer.WriteString("name", name) writer.WritePropertyName("itemType"); writeType writer itemType; writer.WritePropertyName("source"); writeExpression writer source @@ -387,6 +392,7 @@ module IrArtifact = | "intrinsic" -> Lir.Intrinsic(stringProperty "name" element) | "host-external" -> Lir.HostExternal(stringProperty "name" element, (property "arity" element).GetInt32()) | "runtime-context" -> Lir.RuntimeContext + | "target-import" -> Lir.TargetImport(stringProperty "modulePath" element, stringProperty "memberName" element) | "for-each" -> Lir.ForEach(Lir.SymbolId((property "symbol" element).GetInt32()), stringProperty "name" element, readType (property "itemType" element), readExpression (property "source" element), readExpression (property "body" element)) | "type-token" -> Lir.TypeToken(readType (property "tokenType" element)) diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index a59a937..c6121f7 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -51,6 +51,7 @@ module Lir = | Intrinsic of string | HostExternal of string * int | RuntimeContext + | TargetImport of string * string | ForEach of SymbolId * string * Type * Expression * Expression | TypeToken of Type @@ -173,6 +174,7 @@ module LirLowering = | Hir.Intrinsic name -> Lir.Intrinsic name | Hir.HostExternal(name, arity) -> Lir.HostExternal(name, arity) | Hir.RuntimeContext -> Lir.RuntimeContext + | Hir.TargetImport(modulePath, memberName) -> Lir.TargetImport(modulePath, memberName) | Hir.ForEach(id, name, itemType, source, body) -> Lir.ForEach(symbol id, name, itemType, expression source, expression body) | Hir.TypeToken tokenType -> Lir.TypeToken tokenType { Type = value.Type; Span = value.Span; Node = node } diff --git a/src/FScript.Interpreter.Lir/LirInterpreter.fs b/src/FScript.Interpreter.Lir/LirInterpreter.fs index 1730344..4dfa634 100644 --- a/src/FScript.Interpreter.Lir/LirInterpreter.fs +++ b/src/FScript.Interpreter.Lir/LirInterpreter.fs @@ -22,6 +22,7 @@ type LirValue = | VExternal of string * int * LirValue list | VTypeToken of Type | VRuntimeContext + | VTargetImport of string * string * int * LirValue list | VClosure of Lir.SymbolId * string * Lir.Expression * Map ref type ExecutionResult = { Value: LirValue } @@ -64,6 +65,7 @@ module LirInterpreter = | VExternal _ -> "" | VTypeToken value -> $"" | VRuntimeContext -> "" + | VTargetImport _ -> "" let private fail code span message = Error [ { Code = code; Stage = Interpreter; Message = message; Span = Some span } ] @@ -73,6 +75,8 @@ module LirInterpreter = let rec count = function TFun(_, result) -> 1 + count result | _ -> 0 count valueType + let rec private arityFromType = function TFun(_, result) -> 1 + arityFromType result | _ -> 0 + let private invokeIntrinsic apply span name arguments = let unsupported () = fail "FSI3020" span $"Intrinsic '{name}' is not yet executable in LIR." let expectBool = function Ok(VBool value) -> Ok value | Ok _ -> fail "FSI3022" span $"Intrinsic '{name}' callback must return bool." | Error errors -> Error errors @@ -397,6 +401,7 @@ module LirInterpreter = else Ok(VIntrinsic(name, [])) | Lir.HostExternal(name, arity) -> Ok(VExternal(name, arity, [])) | Lir.RuntimeContext -> Ok VRuntimeContext + | Lir.TargetImport(modulePath, memberName) -> Ok(VTargetImport(modulePath, memberName, arityFromType expression.Type, [])) | Lir.ForEach(id, _, _, source, body) -> match evaluate environment source with | Ok(VList values) -> @@ -420,6 +425,10 @@ module LirInterpreter = let arguments = existing @ [ value ] if arguments.Length = arity then fail "FSI3021" span $"Host external '{name}' is not executable from LIR." else Ok(VExternal(name, arity, arguments)) + | VTargetImport(modulePath, memberName, arity, existing) -> + let arguments = existing @ [ value ] + if arguments.Length = arity then fail "FSI3023" span $"Target import '{modulePath}.{memberName}' is not executable from LIR." + else Ok(VTargetImport(modulePath, memberName, arity, arguments)) | _ -> fail "FSI3003" span "Attempted to call a non-function value." and private evaluateMany environment items = items |> List.map (evaluate environment) |> collect diff --git a/src/FScript.Language/Ast.fs b/src/FScript.Language/Ast.fs index ce85299..653f016 100644 --- a/src/FScript.Language/Ast.fs +++ b/src/FScript.Language/Ast.fs @@ -20,6 +20,16 @@ type Param = Annotation: TypeRef option Span: Span } +type Attribute = + { Name: string + Arguments: string list + Span: Span } + +type BackendImport = + { ModulePath: string + MemberName: string + Span: Span } + type Literal = | LInt of int64 | LFloat of float @@ -87,6 +97,7 @@ and Stmt = | SLet of string * Param list * TypeRef option * Expr * bool * bool * Span | SLetPattern of Pattern * Expr * bool * Span | SLetRecGroup of (string * Param list * TypeRef option * Expr * Span) list * bool * Span + | SBackendImport of string * TypeRef * BackendImport * Span | SExpr of Expr type Program = Stmt list diff --git a/src/FScript.Language/Eval.fs b/src/FScript.Language/Eval.fs index a301271..f96937b 100644 --- a/src/FScript.Language/Eval.fs +++ b/src/FScript.Language/Eval.fs @@ -802,6 +802,14 @@ module Eval = match stmt with | TypeInfer.TSType _ -> () + | TypeInfer.TSBackendImport(name, importedType, import, span) -> + let rec arity = function TFun(_, result) -> 1 + arity result | _ -> 0 + let external = + { Name = name + Scheme = Forall([], importedType) + Arity = arity importedType + Impl = fun _ _ -> raise (EvalException { Message = $"Backend import '{import.ModulePath}.{import.MemberName}' is not executable in the legacy evaluator"; Span = span }) } + env <- Map.add name (VExternal(external, [])) env | TypeInfer.TSLet(name, expr, _, isRec, _, span) -> if isRec then match expr with diff --git a/src/FScript.Language/IncludeResolver.fs b/src/FScript.Language/IncludeResolver.fs index 9658a58..393b71c 100644 --- a/src/FScript.Language/IncludeResolver.fs +++ b/src/FScript.Language/IncludeResolver.fs @@ -551,6 +551,8 @@ module IncludeResolver = let bound = rewrittenArgs |> List.fold (fun s p -> Set.add p.Name s) names rewrittenName, rewrittenArgs, rewrittenReturnAnnotation, rewriteExpr bound valueExpr, bindingSpan) SLetRecGroup(rewrittenBindings, isExported, span) + | SBackendImport(name, signature, import, span) -> + SBackendImport(qualifySelfName name, rewriteTypeRef aliases selfPrefix topLevelTypeNames signature, import, span) | SExpr expr -> SExpr (rewriteExpr Set.empty expr) | SImport _ -> failwith "Unexpected unresolved import directive") diff --git a/src/FScript.Language/Interactive.fs b/src/FScript.Language/Interactive.fs index 492c816..385d559 100644 --- a/src/FScript.Language/Interactive.fs +++ b/src/FScript.Language/Interactive.fs @@ -55,6 +55,7 @@ module Interactive = groupBindings |> List.fold (fun state (name, _, bindingType, _) -> state |> Map.add name (inferenceBinding name bindingType)) current + | TypeInfer.TSBackendImport(name, importedType, _, _) -> current |> Map.add name (inferenceBinding name importedType) | TypeInfer.TSType _ | TypeInfer.TSExpr _ -> current) bindings diff --git a/src/FScript.Language/Parser.fs b/src/FScript.Language/Parser.fs index 7688226..394b35f 100644 --- a/src/FScript.Language/Parser.fs +++ b/src/FScript.Language/Parser.fs @@ -1451,6 +1451,8 @@ module Parser = raise (ParseException { Message = "Block cannot end with a let binding; add a final expression"; Span = span }) | [SLetRecGroup(_, _, span)] -> raise (ParseException { Message = "Block cannot end with a let binding; add a final expression"; Span = span }) + | SBackendImport(_, _, _, span) :: _ -> + raise (ParseException { Message = "Backend imports are only supported at top level"; Span = span }) | SLet(name, args, returnAnnotation, value, isRec, _, span) :: rest -> let valExpr = Seq.foldBack (fun arg acc -> ELambda(arg, acc, span)) args value let body = desugar rest @@ -1554,7 +1556,7 @@ module Parser = | None -> SLet(bindingName, args |> Seq.toList, returnAnnotation, value, false, isExported, mkSpanFrom letTok.Span (Ast.spanOfExpr value)) - and parseAttributeName () = + and parseAttribute () = stream.Expect(LBracket, "Expected '[' in attribute") |> ignore stream.Expect(Less, "Expected '<' in attribute") |> ignore let nameTok = stream.ExpectIdent("Expected attribute name") @@ -1562,9 +1564,36 @@ module Parser = match nameTok.Kind with | Ident n -> n | _ -> "" + let arguments = ResizeArray() + if stream.Match(LParen) then + let mutable doneArguments = false + while not doneArguments do + let argument = stream.Next() + match argument.Kind with + | StringLit value | Ident value -> arguments.Add(value) + | _ -> raise (ParseException { Message = "Attribute arguments must be strings or identifiers"; Span = argument.Span }) + if stream.Match(Comma) then () + else + stream.Expect(RParen, "Expected ')' after attribute arguments") |> ignore + doneArguments <- true stream.Expect(Greater, "Expected '>' in attribute") |> ignore stream.Expect(RBracket, "Expected ']' in attribute") |> ignore - name, nameTok.Span + { Name = name; Arguments = arguments |> Seq.toList; Span = nameTok.Span } + + and parseBackendImport (attribute: Attribute) = + let letToken = stream.Expect(Let, "Expected 'let' after Import attribute") + let nameToken = stream.ExpectIdent("Expected imported binding name") + let name = match nameToken.Kind with Ident value -> value | _ -> "" + stream.Expect(Colon, "Imported binding requires a complete type signature") |> ignore + let signature = parseTypeRef() + if stream.Peek().Kind = Equals then + raise (ParseException { Message = "Imported bindings must not have a body"; Span = stream.Peek().Span }) + match attribute.Arguments with + | [ modulePath; memberName ] when not (System.String.IsNullOrWhiteSpace modulePath) && not (System.String.IsNullOrWhiteSpace memberName) -> + let span = mkSpanFrom letToken.Span nameToken.Span + SBackendImport(name, signature, { ModulePath = modulePath; MemberName = memberName; Span = attribute.Span }, span) + | _ -> + raise (ParseException { Message = "Import requires module path and member name: []"; Span = attribute.Span }) and parseStmt () : Stmt = stream.SkipNewlines() @@ -1574,18 +1603,25 @@ module Parser = let mutable isExported = false let mutable hasAttributes = false + let mutable backendImport: Attribute option = None let mutable parsingAttributes = true while parsingAttributes do if stream.Peek().Kind = LBracket && stream.PeekAt(1).Kind = Less then hasAttributes <- true - let name, span = parseAttributeName() - match name with + let attribute = parseAttribute() + match attribute.Name with | "export" -> if isExported then - raise (ParseException { Message = "Duplicate attribute 'export'"; Span = span }) + raise (ParseException { Message = "Duplicate attribute 'export'"; Span = attribute.Span }) + if not attribute.Arguments.IsEmpty then + raise (ParseException { Message = "Export does not accept arguments"; Span = attribute.Span }) isExported <- true + | "Import" -> + if backendImport.IsSome then + raise (ParseException { Message = "Duplicate attribute 'Import'"; Span = attribute.Span }) + backendImport <- Some attribute | _ -> - raise (ParseException { Message = $"Unknown attribute '{name}'"; Span = span }) + raise (ParseException { Message = $"Unknown attribute '{attribute.Name}'"; Span = attribute.Span }) stream.SkipNewlines() else parsingAttributes <- false @@ -1627,7 +1663,10 @@ module Parser = raise (ParseException { Message = "'[]' is only valid for top-level let bindings"; Span = stream.Peek().Span }) parseTypeDecl() | Let -> - parseLetStmt isExported + match backendImport with + | Some attribute when isExported -> raise (ParseException { Message = "Import and Export cannot be combined initially"; Span = attribute.Span }) + | Some attribute -> parseBackendImport attribute + | None -> parseLetStmt isExported | _ when hasAttributes -> raise (ParseException { Message = "'[]' must be followed by a top-level let binding"; Span = stream.Peek().Span }) | _ -> diff --git a/src/FScript.Language/TypeInfer.fs b/src/FScript.Language/TypeInfer.fs index 187a399..c951321 100644 --- a/src/FScript.Language/TypeInfer.fs +++ b/src/FScript.Language/TypeInfer.fs @@ -196,6 +196,7 @@ module TypeInfer = | TSLet of string * Expr * Type * bool * bool * Span | TSLetPattern of Pattern * Expr * Map * bool * Span | TSLetRecGroup of (string * Expr * Type * Span) list * bool * Span + | TSBackendImport of string * Type * BackendImport * Span | TSExpr of TypedExpr type TypedProgram = TypedStmt list @@ -1272,6 +1273,13 @@ module TypeInfer = typed.Add(TSType def) | SImport (_, _, span) -> raise (TypeException { Message = "'import' must be resolved before type inference"; Span = span }) + | SBackendImport(name, signature, import, span) -> + let importedType = annotationTypeFromRef typeDefs span signature + match importedType with + | TFun _ -> + env <- Map.add name (Forall([], importedType)) env + typed.Add(TSBackendImport(name, importedType, import, span)) + | _ -> raise (TypeException { Message = "Backend imports require a complete function signature"; Span = span }) | SLet(name, args, returnAnnotation, expr, isRec, isExported, span) -> let exprVal = Seq.foldBack (fun arg acc -> ELambda(arg, acc, span)) args expr let startIndex = @@ -1526,6 +1534,8 @@ module TypeInfer = | TSLetRecGroup(bindings, _, _) -> for name, _, inferred, span in bindings do globalEnvironment <- Map.add name (allocate name DeclarationSymbol span (Some(Types.generalize Map.empty inferred))) globalEnvironment + | TSBackendImport(name, importedType, _, span) -> + globalEnvironment <- Map.add name (allocate name DeclarationSymbol span (Some(Forall([], importedType)))) globalEnvironment | TSExpr _ -> () let recordReference environment name expression span = environment |> Map.tryFind name |> Option.iter (fun id -> @@ -1590,6 +1600,7 @@ module TypeInfer = | TSLetPattern(pattern, expression, _, _, _) -> visit globalEnvironment expression; bindPattern PatternBindingSymbol globalEnvironment pattern |> ignore | TSLetRecGroup(bindings, _, _) -> bindings |> List.iter (fun (_, expression, _, _) -> visit globalEnvironment expression) | TSExpr expression -> visit globalEnvironment expression.Expr + | TSBackendImport _ -> () | TSType _ -> () { TypedProgram = typed ExpressionTypes = expressions diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index e025cc1..da534df 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -52,6 +52,7 @@ type GoBackendTests() = [ true\n | _ -> false", "true")>] [] [] + [] let upper : string -> string\nupper \"hello\"", "HELLO")>] member _.``Generated projects run with Go`` (source: string, expected: string) = let input = { Program = Parser.parseProgram source; Externs = [] } let compiled = diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index 6de98ed..94bf87a 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -93,7 +93,8 @@ type IrArtifactTests() = "(let rec even n = if n = 0 then true else odd (n - 1)\nand odd n = if n = 0 then false else even (n - 1)\neven 10\n)" "match { [\"a\"] = 1 } with\n | { [key] = value; ..tail } -> value\n | {} -> 0" "match {| Name = \"Ada\" |} with\n | {| Name: string |} -> true\n | _ -> false" - "List.length Env.Arguments" ] do + "List.length Env.Arguments" + "[] let upper : string -> string\nupper \"hello\"" ] do let bytes = compile source |> _.Lir |> IrArtifact.serialize match IrArtifact.deserialize bytes with | Ok _ -> () diff --git a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs index 7134695..7723e60 100644 --- a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs +++ b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs @@ -219,3 +219,19 @@ type LirInterpreterTests() = | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VTuple [ LirValue.VOption(Some(LirValue.VString "sample.fss")); LirValue.VList [ LirValue.VString "one"; LirValue.VString "two" ] ])) | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``Rejects only reachable target imports`` () = + let compile source = Compiler.compile { Program = Parser.parseProgram source; Externs = [] } + match compile "[] let upper : string -> string\n1" with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 1L)) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + match compile "[] let upper : string -> string\nupper \"hello\"" with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok _ -> Assert.Fail("Expected target-import diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSI3023")) diff --git a/tests/FScript.Language.Tests/ParserTests.fs b/tests/FScript.Language.Tests/ParserTests.fs index 5b61bae..cfe0884 100644 --- a/tests/FScript.Language.Tests/ParserTests.fs +++ b/tests/FScript.Language.Tests/ParserTests.fs @@ -809,3 +809,16 @@ type ParserTests () = member _.``Rejects export attribute in nested expression block`` () = let act () = Helpers.parse "(let x =\n [] let y = 1\n y\n)" |> ignore act |> should throw typeof + + [] + member _.``Parses bodyless backend import attributes`` () = + match Helpers.parse "[] let upper : string -> string" with + | [ SBackendImport("upper", TRFun(TRName "string", TRName "string"), import, _) ] -> + import.ModulePath |> should equal "strings" + import.MemberName |> should equal "ToUpper" + | _ -> Assert.Fail("Expected backend import") + + [] + member _.``Rejects backend imports with bodies`` () = + let act () = Helpers.parse "[] let upper : string -> string = fun value -> value" |> ignore + act |> should throw typeof From 783d77b57032f958a64949b5c9507406b2a89801 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 18:07:08 +0200 Subject: [PATCH 028/133] test(go): enforce FScript semantic conformance --- CHANGELOG.md | 1 + FScript.sln | 15 +++ docs/architecture/typed-ir-and-backends.md | 4 + docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Compiler/HirLowering.fs | 16 +++- src/FScript.Compiler/Lir.fs | 3 +- .../ConformanceTests.fs | 94 +++++++++++++++++++ .../FScript.Conformance.Tests.fsproj | 22 +++++ tests/FScript.Conformance.Tests/Program.fs | 4 + 9 files changed, 157 insertions(+), 3 deletions(-) create mode 100644 tests/FScript.Conformance.Tests/ConformanceTests.fs create mode 100644 tests/FScript.Conformance.Tests/FScript.Conformance.Tests.fsproj create mode 100644 tests/FScript.Conformance.Tests/Program.fs diff --git a/CHANGELOG.md b/CHANGELOG.md index 0c8273e..944bc96 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added a serialized-LIR/legacy-evaluator/generated-Go conformance suite and corrected named-record representation in generated functions. - Added bodyless `[]` declarations with typed HIR/LIR linkage and primitive Go package bindings. - Added compiler runtime-context IR so LIR and generated Go programs receive `Env.ScriptName` and executable arguments without host linkage. - Added versioned `.fsir` compiler/language metadata, normalized source SHA-256 records, ordered dependencies, and strict metadata validation. diff --git a/FScript.sln b/FScript.sln index 07d3c6f..5294311 100644 --- a/FScript.sln +++ b/FScript.sln @@ -43,6 +43,8 @@ Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "FScript.Interpreter.Lir.Tes EndProject Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "FScript.Backend.Go.Tests", "tests\FScript.Backend.Go.Tests\FScript.Backend.Go.Tests.fsproj", "{966D205D-F585-4294-9486-72F317ADE7D4}" EndProject +Project("{F2A71F9B-5D33-465A-A702-920D77279786}") = "FScript.Conformance.Tests", "tests\FScript.Conformance.Tests\FScript.Conformance.Tests.fsproj", "{F0792787-B775-4E65-8291-7F2AC2EF097B}" +EndProject Global GlobalSection(SolutionConfigurationPlatforms) = preSolution Debug|Any CPU = Debug|Any CPU @@ -269,6 +271,18 @@ Global {966D205D-F585-4294-9486-72F317ADE7D4}.Release|x64.Build.0 = Release|Any CPU {966D205D-F585-4294-9486-72F317ADE7D4}.Release|x86.ActiveCfg = Release|Any CPU {966D205D-F585-4294-9486-72F317ADE7D4}.Release|x86.Build.0 = Release|Any CPU + {F0792787-B775-4E65-8291-7F2AC2EF097B}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {F0792787-B775-4E65-8291-7F2AC2EF097B}.Debug|Any CPU.Build.0 = Debug|Any CPU + {F0792787-B775-4E65-8291-7F2AC2EF097B}.Debug|x64.ActiveCfg = Debug|Any CPU + {F0792787-B775-4E65-8291-7F2AC2EF097B}.Debug|x64.Build.0 = Debug|Any CPU + {F0792787-B775-4E65-8291-7F2AC2EF097B}.Debug|x86.ActiveCfg = Debug|Any CPU + {F0792787-B775-4E65-8291-7F2AC2EF097B}.Debug|x86.Build.0 = Debug|Any CPU + {F0792787-B775-4E65-8291-7F2AC2EF097B}.Release|Any CPU.ActiveCfg = Release|Any CPU + {F0792787-B775-4E65-8291-7F2AC2EF097B}.Release|Any CPU.Build.0 = Release|Any CPU + {F0792787-B775-4E65-8291-7F2AC2EF097B}.Release|x64.ActiveCfg = Release|Any CPU + {F0792787-B775-4E65-8291-7F2AC2EF097B}.Release|x64.Build.0 = Release|Any CPU + {F0792787-B775-4E65-8291-7F2AC2EF097B}.Release|x86.ActiveCfg = Release|Any CPU + {F0792787-B775-4E65-8291-7F2AC2EF097B}.Release|x86.Build.0 = Release|Any CPU EndGlobalSection GlobalSection(SolutionProperties) = preSolution HideSolutionNode = FALSE @@ -292,5 +306,6 @@ Global {FC0FF53F-A10D-40B2-9C30-87AF93AD8A52} = {827E0CD3-B72D-47B6-A68D-7590B98EB39B} {B9A3B895-A44F-4D1F-BFB9-4E1567885E64} = {0AB3BF05-4346-4AA6-1389-037BE0695223} {966D205D-F585-4294-9486-72F317ADE7D4} = {0AB3BF05-4346-4AA6-1389-037BE0695223} + {F0792787-B775-4E65-8291-7F2AC2EF097B} = {0AB3BF05-4346-4AA6-1389-037BE0695223} EndGlobalSection EndGlobal diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index a880d50..65f85fa 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -68,6 +68,10 @@ The LIR interpreter rejects reachable target imports it cannot supply. The Go ba Initial Go imports use `[] let name : complete -> signature` with no body. The frontend requires a function type and records linkage separately from its FScript signature. Go emission uses deterministic package aliases and currently accepts only primitive/unit arguments and results; other representations receive `FSB4003`. The LIR interpreter keeps unused imports inert and reports `FSI3023` only when one is fully applied. +## Cross-runtime conformance + +The conformance suite evaluates one source program through the legacy evaluator, a serialized-and-deserialized LIR program, and a generated Go executable. It compares successful results and failure categories across nested structural equality, named/structural record conversion, unions, higher-order and polymorphic functions, mutual recursion, and ordered maps. Named semantic types are resolved to their record or union representation while HIR is built, so a named record crossing a function boundary cannot accidentally receive the union representation in a downstream consumer. + ## Previous experiment The `feature/in-memory-execution` branch is useful only as evidence about closures, invocation, and host integration. Its dynamic-method/in-memory assembly backend is not an IR layer and is neither copied nor referenced by the new compiler architecture. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 3214324..69837ba 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -41,6 +41,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Compiled map patterns consume entries deterministically in map order, merge repeated bindings only when structurally equal, and bind an optional tail to the remaining ordered map. Compiled type-reference patterns use resolved types rather than source text lookup. - Record updates evaluate the base once and then updates in source order. Ranges, loops, interpolation, indexing, raise, append, and cons preserve the evaluator's existing evaluation order and failures. - Equality is structural for primitives, tuples, records, maps, options, lists, unions, and type tokens, recursively. Functions, tasks, constructors, and extern handles are not structurally equal. Backend host-language comparability must not leak into this behavior. +- Cross-runtime tests compare the legacy evaluator with artifact-round-tripped LIR execution and generated Go for successful values and failure categories. Named records passed through typed function boundaries keep record field projection semantics; their nominal annotation must not select a union runtime representation. ## Builtins, externs, attributes, and tasks diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index e420ff8..1f060e0 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -15,8 +15,22 @@ module HirLowering = state.NextSymbol <- value + 1 Hir.SymbolId value + let rec private representationType state = function + | TFun(argument, result) -> TFun(representationType state argument, representationType state result) + | TTuple items -> TTuple(List.map (representationType state) items) + | TList item -> TList(representationType state item) + | TOption item -> TOption(representationType state item) + | TMap(key, value) -> TMap(representationType state key, representationType state value) + | TTask item -> TTask(representationType state item) + | TRecord fields -> TRecord(fields |> Map.map (fun _ -> representationType state)) + | TNamed name -> state.TypeDefinitions |> Map.tryFind name |> Option.defaultValue (TNamed name) + | value -> value + let private expressionType state expression fallback = - state.ExpressionTypes |> Map.tryFind expression |> Option.defaultValue fallback + state.ExpressionTypes + |> Map.tryFind expression + |> Option.defaultValue fallback + |> representationType state let private literal = function | LInt value -> Hir.Int value, TInt diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index c6121f7..3b3dbfa 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -183,8 +183,7 @@ module LirLowering = { Declarations = program.Declarations |> List.map (fun value -> - let (Forall(_, declarationType)) = value.Scheme - ({ Id = symbol value.Id; Name = value.Name; Type = declarationType + ({ Id = symbol value.Id; Name = value.Name; Type = value.Value.Type IsRecursive = value.IsRecursive; IsExported = value.IsExported Span = value.Span; Value = expression value.Value }: Lir.Declaration)) Entry = program.Entry |> Option.map expression } diff --git a/tests/FScript.Conformance.Tests/ConformanceTests.fs b/tests/FScript.Conformance.Tests/ConformanceTests.fs new file mode 100644 index 0000000..99a30c1 --- /dev/null +++ b/tests/FScript.Conformance.Tests/ConformanceTests.fs @@ -0,0 +1,94 @@ +namespace FScript.Conformance.Tests + +open System +open System.Diagnostics +open System.IO +open FScript.Compiler +open FScript.Language +open NUnit.Framework + +module private Harness = + let rec formatLir = function + | LirValue.VUnit -> "()" + | LirValue.VInt value -> string value + | LirValue.VFloat value -> value.ToString("R", Globalization.CultureInfo.InvariantCulture) + | LirValue.VBool value -> string value |> _.ToLowerInvariant() + | LirValue.VString value -> value + | LirValue.VOption None -> "None" + | LirValue.VOption(Some value) -> "Some " + formatLir value + | LirValue.VList values -> "[" + (values |> List.map formatLir |> String.concat "; ") + "]" + | LirValue.VTuple values -> "(" + (values |> List.map formatLir |> String.concat ", ") + ")" + | LirValue.VRecord fields -> "{ " + (fields |> Map.toList |> List.map (fun (name, value) -> name + " = " + formatLir value) |> String.concat "; ") + " }" + | LirValue.VUnion(_, caseName, None) -> caseName + | LirValue.VUnion(_, caseName, Some value) -> caseName + " " + formatLir value + | LirValue.VMap entries -> "{ " + (entries |> List.map (fun (key, value) -> "[" + formatLir key + "] = " + formatLir value) |> String.concat "; ") + " }" + | LirValue.VTypeToken value -> "" + | LirValue.VClosure _ | LirValue.VIntrinsic _ | LirValue.VExternal _ | LirValue.VTargetImport _ -> "" + | LirValue.VRuntimeContext -> "" + + let compile source = + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Ok value -> value + | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + + let legacy source = source |> Parser.parseProgram |> TypeInfer.inferProgram |> Eval.evalProgram |> Pretty.valueToString + + let lir source = + let artifact = compile source |> _.Lir |> IrArtifact.serialize + let loaded = match IrArtifact.deserialize artifact with Ok value -> value | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + match LirInterpreter.execute { ScriptName = None; Arguments = [] } loaded with + | Ok result -> Ok(formatLir result.Value) + | Error diagnostics -> Error diagnostics.Head.Code + + let go source = + let files = match GoBackend.generate { ModuleName = "fscript.generated/conformance" } (compile source).Lir with Ok value -> value | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + let directory = Path.Combine(Path.GetTempPath(), $"fscript-conformance-{Guid.NewGuid():N}") + try + for file in files do + let path = Path.Combine(directory, file.RelativePath) + match Path.GetDirectoryName(path) with + | null -> () + | parent -> Directory.CreateDirectory(parent) |> ignore + File.WriteAllText(path, file.Content) + let start = ProcessStartInfo("go") + start.WorkingDirectory <- directory; start.ArgumentList.Add("run"); start.ArgumentList.Add(".") + start.RedirectStandardOutput <- true; start.RedirectStandardError <- true + use proc = + match Process.Start(start) with + | null -> failwith "Could not start the Go toolchain" + | started -> started + let output = proc.StandardOutput.ReadToEnd().Trim() + let error = proc.StandardError.ReadToEnd() + proc.WaitForExit() + if proc.ExitCode = 0 then Ok output else Error error + finally + if Directory.Exists directory then Directory.Delete(directory, true) + +[] +type ConformanceTests() = + [] + [] + [ 0 | Circle radius -> radius")>] + [] + [ state + value) 0 [1; 2; 3]")>] + [] + [ value | None -> 0")>] + member _.``Legacy LIR artifact and Go results conform`` (source: string) = + let expected = Harness.legacy source + match Harness.lir source with + | Ok actual -> Assert.That(actual, Is.EqualTo(expected), source) + | Error code -> Assert.Fail($"LIR failed with {code}: {source}") + match Harness.go source with + | Ok actual -> Assert.That(actual, Is.EqualTo(expected), source) + | Error error -> Assert.Fail($"Go failed with {error}: {source}") + + [] + member _.``Failure categories conform`` () = + let source = "raise \"boom\"" + Assert.Throws(fun () -> Harness.legacy source |> ignore) |> ignore + match Harness.lir source with + | Error code -> Assert.That(code, Is.EqualTo("FSI3011")) + | Ok value -> Assert.Fail($"Expected LIR failure, got {value}") + match Harness.go source with + | Ok value -> Assert.Fail($"Expected generated Go failure, got {value}") + | Error error -> Assert.That(error, Does.Contain("boom")) diff --git a/tests/FScript.Conformance.Tests/FScript.Conformance.Tests.fsproj b/tests/FScript.Conformance.Tests/FScript.Conformance.Tests.fsproj new file mode 100644 index 0000000..2df36eb --- /dev/null +++ b/tests/FScript.Conformance.Tests/FScript.Conformance.Tests.fsproj @@ -0,0 +1,22 @@ + + + net10.0 + false + false + + + + + + + + + + + + + + + + + diff --git a/tests/FScript.Conformance.Tests/Program.fs b/tests/FScript.Conformance.Tests/Program.fs new file mode 100644 index 0000000..b470242 --- /dev/null +++ b/tests/FScript.Conformance.Tests/Program.fs @@ -0,0 +1,4 @@ +module FScript.Conformance.Tests.Program + +[] +let main _ = 0 From 134683c640339f5901b0d73db4bcd29906d0641d Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 18:13:12 +0200 Subject: [PATCH 029/133] feat(compiler): validate typed IR invariants --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 + docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Compiler/HirValidation.fs | 23 ++++ src/FScript.Compiler/Lir.fs | 133 ++++++++++++++++++++- tests/FScript.Compiler.Tests/HirTests.fs | 29 +++++ 6 files changed, 188 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 944bc96..c75e072 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added HIR/LIR validation for declaration schemes, source spans, representations, calls, control flow, references, imports, and primitive availability. - Added a serialized-LIR/legacy-evaluator/generated-Go conformance suite and corrected named-record representation in generated functions. - Added bodyless `[]` declarations with typed HIR/LIR linkage and primitive Go package bindings. - Added compiler runtime-context IR so LIR and generated Go programs receive `Env.ScriptName` and executable arguments without host linkage. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 65f85fa..0aa0535 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -24,6 +24,8 @@ HIR is language-oriented and contains resolved artifact-local symbol identities, LIR is executable but not a Go AST. It uses lifted functions, explicit captures and closure creation, resolved primitive operations, aggregate construction/projection, and structured control flow. Pattern matching becomes deterministic tests and projections before reaching a backend. Semantic types may remain alongside selected runtime representations for validation and diagnostics. +HIR validation rejects duplicate and unresolved symbols, invalid declaration spans, and schemes inconsistent with their values. LIR validation rechecks artifact-local references and spans, then verifies closure calls, branches and decision guards, primitive signatures, aggregate representations, declaration value types, intrinsic catalog membership, and complete target-import linkage. Artifact deserialization runs the same LIR validator, so malformed JSON cannot bypass compiler invariants. + Lexical `let` is retained as a structured LIR node. Its symbol is visible only in the body, or in both the binding and body for recursive functions. The interpreter closes the recursive environment explicitly; Go emission declares the function slot before assigning its closure. `raise` becomes an explicit `Fail` node so consumers preserve evaluation order and attach their own stage-specific failure diagnostic. Matches lower to a structured decision tree whose target is evaluated once. Cases remain ordered; each pattern test creates artifact-local bindings, then its guard is evaluated, and only then is its body selected. A failed guard continues at the next case. Exhaustion is the explicit FScript match-failure path. Literal, list, tuple, record, and option patterns use the same tree in serialized artifacts, the LIR interpreter, and generated Go. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 69837ba..91f1dc4 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -42,6 +42,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Record updates evaluate the base once and then updates in source order. Ranges, loops, interpolation, indexing, raise, append, and cons preserve the evaluator's existing evaluation order and failures. - Equality is structural for primitives, tuples, records, maps, options, lists, unions, and type tokens, recursively. Functions, tasks, constructors, and extern handles are not structurally equal. Backend host-language comparability must not leak into this behavior. - Cross-runtime tests compare the legacy evaluator with artifact-round-tripped LIR execution and generated Go for successful values and failure categories. Named records passed through typed function boundaries keep record field projection semantics; their nominal annotation must not select a union runtime representation. +- HIR and LIR are accepted only when artifact-local references resolve, source spans are valid, declaration and expression types agree, calls and control-flow nodes have consistent signatures, aggregate representations match their semantic types, and referenced primitives/imports are available and complete. ## Builtins, externs, attributes, and tasks diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index 3c0173d..36c5980 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -1,11 +1,34 @@ namespace FScript.Compiler +open FScript.Language + module HirValidation = + let rec private compatible left right = + match left, right with + | TVar _, _ | _, TVar _ -> true + | TFun(la, lr), TFun(ra, rr) -> compatible la ra && compatible lr rr + | TTuple left, TTuple right -> left.Length = right.Length && List.forall2 compatible left right + | TList left, TList right | TOption left, TOption right | TTask left, TTask right -> compatible left right + | TMap(lk, lv), TMap(rk, rv) -> compatible lk rk && compatible lv rv + | TRecord left, TRecord right -> left.Count = right.Count && (left |> Map.forall (fun name item -> right |> Map.tryFind name |> Option.exists (compatible item))) + | TUnion(ln, _), TUnion(rn, _) -> ln = rn + | TNamed name, TUnion(actual, _) | TUnion(actual, _), TNamed name -> name = actual + | TNamed _, TRecord _ | TRecord _, TNamed _ -> true + | _ -> left = right + + let private validSpan (span: Span) = + span.Start.Line >= 0 && span.Start.Column >= 0 && span.End.Line >= 0 && span.End.Column >= 0 + && (span.Start.File <> span.End.File || span.End.Line > span.Start.Line || (span.End.Line = span.Start.Line && span.End.Column >= span.Start.Column)) + let validate (program: Hir.Program) = let declarationIds = program.Declarations |> List.map (fun value -> value.Id) let duplicateIds = declarationIds |> List.countBy id |> List.filter (fun (_, count) -> count > 1) if not duplicateIds.IsEmpty then Error [ Diagnostic.hir "FSH1101" "HIR declaration IDs must be unique." None ] + elif program.Declarations |> List.exists (fun declaration -> not (validSpan declaration.Span && validSpan declaration.Value.Span)) then + Error [ Diagnostic.hir "FSH1103" "HIR contains an invalid source span." None ] + elif program.Declarations |> List.exists (fun declaration -> let (Forall(_, declaredType)) = declaration.Scheme in not (compatible declaredType declaration.Value.Type)) then + Error [ Diagnostic.hir "FSH1104" "HIR declaration scheme does not match its value type." None ] else let rec references (expression: Hir.Expression) = match expression.Node with diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index 3b3dbfa..bb5c806 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -189,8 +189,139 @@ module LirLowering = Entry = program.Entry |> Option.map expression } module LirValidation = + let rec private compatible left right = + match left, right with + | TVar _, _ | _, TVar _ -> true + | TFun(la, lr), TFun(ra, rr) -> compatible la ra && compatible lr rr + | TTuple left, TTuple right -> left.Length = right.Length && List.forall2 compatible left right + | TList left, TList right | TOption left, TOption right | TTask left, TTask right -> compatible left right + | TMap(lk, lv), TMap(rk, rv) -> compatible lk rk && compatible lv rv + | TRecord left, TRecord right -> + left.Count = right.Count + && (left |> Map.forall (fun name item -> right |> Map.tryFind name |> Option.exists (compatible item))) + | TUnion(ln, _), TUnion(rn, _) -> ln = rn + | _ -> left = right + + let private literalType = function + | Lir.Unit -> TUnit | Lir.Int _ -> TInt | Lir.Float _ -> TFloat + | Lir.Bool _ -> TBool | Lir.String _ -> TString + let validate (program: Lir.Program) = let ids = program.Declarations |> List.map _.Id if ids.Length <> (ids |> Set.ofList |> Set.count) then Error [ { Code = "FSL2001"; Stage = Lowering; Message = "LIR declaration IDs must be unique."; Span = None } ] - else Ok program + else + let diagnostics = ResizeArray() + let report code message span = + diagnostics.Add({ Code = code; Stage = Lowering; Message = message; Span = Some span }) + let validSpan (span: Span) = + span.Start.Line >= 0 && span.Start.Column >= 0 && span.End.Line >= 0 && span.End.Column >= 0 + && (span.Start.File <> span.End.File + || span.End.Line > span.Start.Line + || (span.End.Line = span.Start.Line && span.End.Column >= span.Start.Column)) + let checkSpan span = if not (validSpan span) then report "FSL2002" "LIR source span is invalid." span + let rec checkPattern expected (pattern: Lir.Pattern) = + checkSpan pattern.Span + if not (compatible expected pattern.Type) then report "FSL2003" "LIR pattern type does not match its scrutinee." pattern.Span + match pattern.Node with + | Lir.LiteralPattern value when not (compatible pattern.Type (literalType value)) -> report "FSL2003" "LIR literal pattern has an inconsistent type." pattern.Span + | Lir.ConsPattern(head, tail) -> + match pattern.Type with + | TList item -> checkPattern item head; checkPattern pattern.Type tail + | _ -> report "FSL2003" "LIR cons pattern requires a list." pattern.Span + | Lir.TuplePattern items -> + match pattern.Type with + | TTuple types when types.Length = items.Length -> List.iter2 checkPattern types items + | _ -> report "FSL2003" "LIR tuple pattern shape is inconsistent." pattern.Span + | Lir.RecordPattern fields -> + match pattern.Type with + | TRecord types -> fields |> List.iter (fun (name, item) -> match Map.tryFind name types with Some itemType -> checkPattern itemType item | None -> report "FSL2003" $"Unknown LIR record-pattern field '{name}'." item.Span) + | _ -> report "FSL2003" "LIR record pattern requires a record." pattern.Span + | Lir.SomePattern item -> match pattern.Type with TOption itemType -> checkPattern itemType item | _ -> report "FSL2003" "LIR some pattern requires an option." pattern.Span + | Lir.EmptyList -> match pattern.Type with TList _ -> () | _ -> report "FSL2003" "LIR empty-list pattern requires a list." pattern.Span + | Lir.NonePattern -> match pattern.Type with TOption _ -> () | _ -> report "FSL2003" "LIR none pattern requires an option." pattern.Span + | Lir.MapPattern(clauses, tail) -> + match pattern.Type with + | TMap(keyType, valueType) -> clauses |> List.iter (fun (key, value) -> checkPattern keyType key; checkPattern valueType value); tail |> Option.iter (checkPattern pattern.Type) + | _ -> report "FSL2003" "LIR map pattern requires a map." pattern.Span + | Lir.UnionPattern(typeName, caseName, payload) -> + match pattern.Type with + | TUnion(actual, cases) when actual = typeName -> + match Map.tryFind caseName cases, payload with + | Some(Some payloadType), Some value -> checkPattern payloadType value + | Some None, None -> () + | _ -> report "FSL2003" "LIR union-pattern payload is inconsistent." pattern.Span + | _ -> report "FSL2003" "LIR union pattern has an inconsistent declared type." pattern.Span + | _ -> () + let rec patternBindings (pattern: Lir.Pattern) = + match pattern.Node with + | Lir.Binding(id, _) -> Set.singleton id + | Lir.ConsPattern(head, tail) -> Set.union (patternBindings head) (patternBindings tail) + | Lir.TuplePattern items -> items |> List.map patternBindings |> Set.unionMany + | Lir.RecordPattern fields -> fields |> List.map (snd >> patternBindings) |> Set.unionMany + | Lir.MapPattern(clauses, tail) -> + [ yield! clauses |> List.collect (fun (key, value) -> [ patternBindings key; patternBindings value ]) + match tail with Some value -> patternBindings value | None -> Set.empty ] |> Set.unionMany + | Lir.SomePattern value -> patternBindings value + | Lir.UnionPattern(_, _, payload) -> payload |> Option.map patternBindings |> Option.defaultValue Set.empty + | _ -> Set.empty + let declared = ids |> Set.ofList + let rec checkExpression locals (expression: Lir.Expression) = + checkSpan expression.Span + let check = checkExpression locals + match expression.Node with + | Lir.Constant value when not (compatible expression.Type (literalType value)) -> report "FSL2004" "LIR constant has an inconsistent type." expression.Span + | Lir.Load id when not (Set.contains id locals || Set.contains id declared) -> report "FSL2005" (sprintf "LIR reference %%%d is unresolved." (Lir.symbolNumber id)) expression.Span + | Lir.MakeClosure(parameter, _, parameterType, body) -> + checkExpression (Set.add parameter locals) body + match expression.Type with TFun(argument, result) when compatible argument parameterType && compatible result body.Type -> () | _ -> report "FSL2006" "LIR closure signature is inconsistent with its body." expression.Span + | Lir.CallClosure(fn, argument) -> + check fn; check argument + match fn.Type with TFun(parameter, result) when compatible parameter argument.Type && compatible result expression.Type -> () | _ -> report "FSL2006" "LIR closure call signature is inconsistent." expression.Span + | Lir.Branch(condition, whenTrue, whenFalse) -> + check condition; check whenTrue; check whenFalse + if not (compatible condition.Type TBool && compatible expression.Type whenTrue.Type && compatible expression.Type whenFalse.Type) then report "FSL2007" "LIR branch types are inconsistent." expression.Span + | Lir.Operation(operation, arguments) -> + arguments |> List.iter check + let expected = + match operation with + | Lir.IntAdd | Lir.IntSubtract | Lir.IntMultiply | Lir.IntDivide | Lir.IntRemainder -> Some(TInt, TInt) + | Lir.FloatAdd | Lir.FloatSubtract | Lir.FloatMultiply | Lir.FloatDivide -> Some(TFloat, TFloat) + | Lir.BooleanAnd | Lir.BooleanOr -> Some(TBool, TBool) + | Lir.StructuralEqual -> None + | _ -> Some((arguments |> List.tryHead |> Option.map _.Type |> Option.defaultValue TInt), TBool) + if arguments.Length <> 2 then report "FSL2008" "LIR primitive operation requires two operands." expression.Span + match expected with Some(operand, result) when not (arguments |> List.forall (fun item -> compatible operand item.Type) && compatible result expression.Type) -> report "FSL2008" "LIR primitive operand or result type is inconsistent." expression.Span | _ -> () + | Lir.Tuple items -> items |> List.iter check; match expression.Type with TTuple types when types.Length = items.Length && List.forall2 (fun expected (item: Lir.Expression) -> compatible expected item.Type) types items -> () | _ -> report "FSL2004" "LIR tuple representation is inconsistent." expression.Span + | Lir.List items -> items |> List.iter check; match expression.Type with TList itemType when items |> List.forall (fun item -> compatible itemType item.Type) -> () | _ -> report "FSL2004" "LIR list representation is inconsistent." expression.Span + | Lir.Record fields -> fields |> List.iter (snd >> check); match expression.Type with TRecord types when fields.Length = types.Count && fields |> List.forall (fun (name, item) -> Map.tryFind name types |> Option.exists (fun expected -> compatible expected item.Type)) -> () | _ -> report "FSL2004" "LIR record representation is inconsistent." expression.Span + | Lir.FieldGet(target, name) -> + check target + match target.Type, name with + | TRecord fields, _ when Map.tryFind name fields |> Option.exists (compatible expression.Type) -> () + | TNamed "Environment", "ScriptName" when compatible expression.Type (TOption TString) -> () + | TNamed "Environment", "Arguments" when compatible expression.Type (TList TString) -> () + | _ -> report "FSL2004" $"LIR field projection '{name}' is inconsistent." expression.Span + | Lir.SomeValue item -> check item; match expression.Type with TOption itemType when compatible itemType item.Type -> () | _ -> report "FSL2004" "LIR option representation is inconsistent." expression.Span + | Lir.NoneValue -> match expression.Type with TOption _ -> () | _ -> report "FSL2004" "LIR none value requires an option type." expression.Span + | Lir.Cons(head, tail) -> check head; check tail; match expression.Type, tail.Type with TList item, TList tailItem when compatible item head.Type && compatible item tailItem -> () | _ -> report "FSL2004" "LIR cons representation is inconsistent." expression.Span + | Lir.Append(left, right) -> check left; check right; if not (compatible expression.Type left.Type && compatible expression.Type right.Type) then report "FSL2004" "LIR append representation is inconsistent." expression.Span + | Lir.Let(id, _, recursive, binding, body) -> let scope = Set.add id locals in checkExpression (if recursive then scope else locals) binding; checkExpression scope body; if not (compatible expression.Type body.Type) then report "FSL2004" "LIR let result type is inconsistent." expression.Span + | Lir.LetRecursiveGroup(bindings, body) -> let scope = bindings |> List.fold (fun value (id, _, _, _) -> Set.add id value) locals in bindings |> List.iter (fun (_, _, bindingType, value) -> checkExpression scope value; if not (compatible bindingType value.Type) then report "FSL2006" "LIR recursive binding signature is inconsistent." value.Span); checkExpression scope body; if not (compatible expression.Type body.Type) then report "FSL2004" "LIR recursive group result is inconsistent." expression.Span + | Lir.Fail value -> check value + | Lir.DecisionTree(target, cases) -> check target; cases |> List.iter (fun case -> let caseLocals = Set.union locals (patternBindings case.Pattern) in checkPattern target.Type case.Pattern; case.Guard |> Option.iter (fun guard -> checkExpression caseLocals guard; if not (compatible guard.Type TBool) then report "FSL2007" "LIR decision guard must be Boolean." guard.Span); checkExpression caseLocals case.Body; if not (compatible expression.Type case.Body.Type) then report "FSL2007" "LIR decision body type is inconsistent." case.Body.Span) + | Lir.Range(first, last) -> check first; check last; if not (compatible first.Type TInt && compatible last.Type TInt && compatible expression.Type (TList TInt)) then report "FSL2004" "LIR range representation is inconsistent." expression.Span + | Lir.RecordUpdate(target, fields) -> check target; fields |> List.iter (snd >> check); if not (compatible expression.Type target.Type) then report "FSL2004" "LIR record update representation is inconsistent." expression.Span + | Lir.IndexGet(target, index) -> check target; check index; match target.Type, expression.Type with TList item, TOption result when compatible index.Type TInt && compatible item result -> () | TMap(key, value), TOption result when compatible key index.Type && compatible value result -> () | _ -> report "FSL2004" "LIR index representation is inconsistent." expression.Span + | Lir.Interpolate parts -> parts |> List.iter (function Lir.TextPart _ -> () | Lir.ValuePart value -> check value); if not (compatible expression.Type TString) then report "FSL2004" "LIR interpolation must produce string." expression.Span + | Lir.UnionCase(typeName, caseName, payload) -> payload |> Option.iter check; match expression.Type with TUnion(actual, cases) when actual = typeName && Map.containsKey caseName cases -> () | _ -> report "FSL2004" "LIR union representation is inconsistent." expression.Span + | Lir.Map entries -> entries |> List.iter (function Lir.KeyValue(key, value) -> check key; check value | Lir.Spread value -> check value); match expression.Type with TMap _ -> () | _ -> report "FSL2004" "LIR map node requires a map type." expression.Span + | Lir.Intrinsic name when not (Map.containsKey name BuiltinSignatures.builtinSchemes) -> report "FSL2009" $"Unknown LIR intrinsic '{name}'." expression.Span + | Lir.HostExternal(_, arity) when arity < 0 -> report "FSL2010" "LIR host external arity cannot be negative." expression.Span + | Lir.TargetImport(modulePath, memberName) when System.String.IsNullOrWhiteSpace(modulePath) || System.String.IsNullOrWhiteSpace(memberName) -> report "FSL2011" "LIR target import linkage is incomplete." expression.Span + | Lir.ForEach(id, _, itemType, source, body) -> check source; checkExpression (Set.add id locals) body; match source.Type with TList actual when compatible actual itemType && compatible expression.Type TUnit -> () | _ -> report "FSL2004" "LIR for-loop representation is inconsistent." expression.Span + | Lir.TypeToken _ when not (compatible expression.Type TTypeToken) -> report "FSL2004" "LIR type token has an inconsistent type." expression.Span + | _ -> () + program.Declarations |> List.iter (fun declaration -> checkSpan declaration.Span; checkExpression Set.empty declaration.Value; if not (compatible declaration.Type declaration.Value.Type) then report "FSL2004" $"LIR declaration '{declaration.Name}' type is inconsistent." declaration.Span) + program.Entry |> Option.iter (checkExpression Set.empty) + if diagnostics.Count = 0 then Ok program else Error(List.ofSeq diagnostics) diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index d397ceb..36b78c0 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -6,6 +6,8 @@ open NUnit.Framework [] type HirTests() = + let span = Span.mk (Span.pos 1 0) (Span.pos 1 1) + [] member _.``Lowers deterministic typed add program`` () = let source = "let add x y = x + y\nadd 1 2" @@ -18,3 +20,30 @@ type HirTests() = Assert.That(actual, Does.Contain("primitive IntAdd")) Assert.That(actual, Does.Contain("entry")) Assert.That(Hir.format compiled.Hir, Is.EqualTo(actual)) + + [] + member _.``HIR validation rejects a declaration scheme mismatch`` () = + let value: Hir.Expression = { Type = TInt; Span = span; Node = Hir.Constant(Hir.Int 1L) } + let declaration: Hir.Declaration = + { Id = Hir.SymbolId 0; Name = "value"; Scheme = Forall([], TString) + IsRecursive = false; IsExported = false; Span = span; Value = value } + match HirValidation.validate { Declarations = [ declaration ]; Entry = None } with + | Ok _ -> Assert.Fail("Expected invalid HIR diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSH1104")) + + [] + member _.``LIR validation rejects inconsistent control flow`` () = + let constant valueType value = ({ Type = valueType; Span = span; Node = Lir.Constant value }: Lir.Expression) + let branch: Lir.Expression = + { Type = TInt; Span = span + Node = Lir.Branch(constant TInt (Lir.Int 1L), constant TInt (Lir.Int 2L), constant TString (Lir.String "no")) } + match LirValidation.validate { Declarations = []; Entry = Some branch } with + | Ok _ -> Assert.Fail("Expected invalid LIR diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSL2007")) + + [] + member _.``LIR validation rejects unknown primitives`` () = + let intrinsic = ({ Type = TInt; Span = span; Node = Lir.Intrinsic "Unknown.value" }: Lir.Expression) + match LirValidation.validate { Declarations = []; Entry = Some intrinsic } with + | Ok _ -> Assert.Fail("Expected unavailable primitive diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSL2009")) From 433cc121746a340ea6930618dd9bf457032fff27 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 18:16:34 +0200 Subject: [PATCH 030/133] feat(compiler): preserve deferred task linkage --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 + docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Backend.Go/GoBackend.fs | 46 ++++++++++++++++++- src/FScript.Compiler/Compiler.fs | 16 +++++-- .../GoBackendTests.fs | 10 ++++ .../LirInterpreterTests.fs | 17 +++++++ 7 files changed, 87 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c75e072..c4b3f7d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Preserved `Task.spawn`/`Task.await` as typed host linkage in compiled IR, with reachable-use diagnostics in LIR execution and Go generation. - Added HIR/LIR validation for declaration schemes, source spans, representations, calls, control flow, references, imports, and primitive availability. - Added a serialized-LIR/legacy-evaluator/generated-Go conformance suite and corrected named-record representation in generated functions. - Added bodyless `[]` declarations with typed HIR/LIR linkage and primitive Go package bindings. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 0aa0535..f47bf70 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -44,6 +44,8 @@ Generated Go represents every FScript function as an erased `fsruntime.Closure` Host externs are copied into IR as name, signature, and arity only; their executable .NET delegates never cross the compiler boundary. Constructing or retaining an unused extern handle is inert. Fully applying one in the LIR interpreter produces `FSI3021`, preserving the rule that unreachable unsupported linkage does not block execution. +`Task.spawn` and `Task.await` are compiler-known host-linkage descriptors with their polymorphic task signatures. This lets task-typed declarations survive inference, HIR, LIR, and artifact serialization. The LIR interpreter reports `FSI3021` only when either descriptor is fully applied; the Go backend performs deterministic reachability over declaration references and reports `FSB4004` when a reachable declaration requires a task representation. + Local binding patterns reuse the same single-scrutinee decision-tree representation as `match`. Sequential `for ... in ... do ...` is explicit in HIR and LIR so each consumer evaluates the input list once, visits values in order, and discards each body result before returning unit. `typeof` resolves declared record and union definitions during HIR construction and carries the semantic type as a serializable type-token node. Every top-level source expression becomes a synthetic declaration at its exact position in the declaration stream; the program entry references the final such value. This preserves eager source ordering without evaluating the result expression twice. Top-level pattern bindings similarly store their scrutinee in one synthetic declaration, validate the pattern, and publish deterministic projections for each bound name. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 91f1dc4..9e903e1 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -43,6 +43,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Equality is structural for primitives, tuples, records, maps, options, lists, unions, and type tokens, recursively. Functions, tasks, constructors, and extern handles are not structurally equal. Backend host-language comparability must not leak into this behavior. - Cross-runtime tests compare the legacy evaluator with artifact-round-tripped LIR execution and generated Go for successful values and failure categories. Named records passed through typed function boundaries keep record field projection semantics; their nominal annotation must not select a union runtime representation. - HIR and LIR are accepted only when artifact-local references resolve, source spans are valid, declaration and expression types agree, calls and control-flow nodes have consistent signatures, aggregate representations match their semantic types, and referenced primitives/imports are available and complete. +- Task signatures and host linkage remain representable in compiled artifacts. An unused task function handle is inert; fully applying `Task.spawn` or `Task.await` in LIR reports `FSI3021`, while a Go program whose reachable declaration graph requires a task reports `FSB4004`. ## Builtins, externs, attributes, and tasks diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index 50efd33..c53d2d2 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -452,7 +452,7 @@ func Environment() map[string]any { } """ - let generate options (program: Lir.Program) : Result = + let private generateCore options (program: Lir.Program) : Result = try let targetImports = program.Declarations @@ -506,3 +506,47 @@ func Environment() map[string]any { { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"sort\"\n\t\"strconv\"\n\t\"strings\"\n)\n\ntype Option struct { HasValue bool; Value any }\ntype Union struct { TypeName string; CaseName string; HasValue bool; Value any }\ntype MapEntry struct { Key any; Value any }\ntype OrderedMap struct { Entries []MapEntry }\ntype MapPart struct { IsSpread bool; Key any; Value any; Map OrderedMap }\n\nfunc Unit(value any) struct{} { return struct{}{} }\nfunc Int(value any) int64 { return value.(int64) }\nfunc Float(value any) float64 { return value.(float64) }\nfunc Bool(value any) bool { return value.(bool) }\nfunc String(value any) string { return value.(string) }\nfunc List(value any) []any { return value.([]any) }\nfunc Record(value any) map[string]any { return value.(map[string]any) }\nfunc AsOption(value any) Option { return value.(Option) }\nfunc AsUnion(value any) Union { return value.(Union) }\nfunc AsMap(value any) OrderedMap { return value.(OrderedMap) }\nfunc Some(value any) Option { return Option{HasValue: true, Value: value} }\nfunc None() Option { return Option{} }\nfunc IsSome(value any) bool { return AsOption(value).HasValue }\nfunc OptionValue(value any) any { return AsOption(value).Value }\nfunc MakeUnion(typeName string, caseName string) Union { return Union{TypeName: typeName, CaseName: caseName} }\nfunc MakeUnionValue(typeName string, caseName string, value any) Union { return Union{TypeName: typeName, CaseName: caseName, HasValue: true, Value: value} }\nfunc IsUnion(value any, typeName string, caseName string) bool { item := AsUnion(value); return item.TypeName == typeName && item.CaseName == caseName }\nfunc UnionHasValue(value any) bool { return AsUnion(value).HasValue }\nfunc UnionValue(value any) any { return AsUnion(value).Value }\nfunc HasField(value any, name string) bool { _, ok := Record(value)[name]; return ok }\nfunc Equal(left any, right any) bool { return reflect.DeepEqual(left, right) }\nfunc Cons(head any, tail []any) []any { return append([]any{head}, tail...) }\nfunc Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) }\nfunc Tail(value any) []any { return append([]any{}, List(value)[1:]...) }\nfunc Range(first int64, last int64) []any { step := int64(1); if first > last { step = -1 }; values := []any{}; for current := first; ; current += step { values = append(values, current); if current == last { break } }; return values }\nfunc UpdateRecord(value any, updates map[string]any) map[string]any { result := map[string]any{}; for key, item := range Record(value) { result[key] = item }; for key, item := range updates { result[key] = item }; return result }\nfunc Pair(key any, value any) MapPart { return MapPart{Key: key, Value: value} }\nfunc SpreadMap(value any) MapPart { return MapPart{IsSpread: true, Map: AsMap(value)} }\nfunc mapSet(entries []MapEntry, key any, value any, replace bool) []MapEntry { for index, entry := range entries { if Equal(entry.Key, key) { if replace { entries[index].Value = value }; return entries } }; return append(entries, MapEntry{Key: key, Value: value}) }\nfunc BuildMap(parts []MapPart) OrderedMap { entries := []MapEntry{}; for _, part := range parts { if part.IsSpread { for _, entry := range part.Map.Entries { entries = mapSet(entries, entry.Key, entry.Value, false) } } else { entries = mapSet(entries, part.Key, part.Value, true) } }; sort.SliceStable(entries, func(left int, right int) bool { return Format(entries[left].Key) < Format(entries[right].Key) }); return OrderedMap{Entries: entries} }\nfunc Index(value any, index any) Option { switch target := value.(type) { case []any: position := Int(index); if position < 0 || position >= int64(len(target)) { return None() }; return Some(target[position]); case OrderedMap: for _, entry := range target.Entries { if Equal(entry.Key, index) { return Some(entry.Value) } }; return None(); default: panic(\"Index access requires a list or map\") } }\nfunc Intrinsic(name string, args []any) any { switch name { case \"ignore\": return struct{}{}; case \"List.empty\": return []any{}; case \"Map.empty\": return OrderedMap{}; case \"Int.tryParse\": value, error := strconv.ParseInt(String(args[0]), 10, 64); if error != nil { return None() }; return Some(value); case \"Float.tryParse\": value, error := strconv.ParseFloat(String(args[0]), 64); if error != nil { return None() }; return Some(value); case \"Bool.tryParse\": value, error := strconv.ParseBool(String(args[0])); if error != nil { return None() }; return Some(value); case \"Int.toString\", \"Float.toString\", \"Bool.toString\": return Format(args[0]); case \"String.replace\": return strings.ReplaceAll(String(args[2]), String(args[0]), String(args[1])); case \"String.indexOf\": index := strings.Index(String(args[1]), String(args[0])); if index < 0 { return None() }; return Some(int64(index)); case \"String.toLower\": return strings.ToLower(String(args[0])); case \"String.toUpper\": return strings.ToUpper(String(args[0])); case \"String.substring\": start, length, value := Int(args[0]), Int(args[1]), String(args[2]); if start < 0 || length < 0 || start + length > int64(len(value)) { return None() }; return Some(value[start:start+length]); case \"String.concat\": values := List(args[1]); parts := make([]string, len(values)); for index, value := range values { parts[index] = String(value) }; return strings.Join(parts, String(args[0])); case \"String.split\": values := strings.Split(String(args[1]), String(args[0])); result := make([]any, len(values)); for index, value := range values { result[index] = value }; return result; case \"String.endsWith\": return strings.HasSuffix(String(args[1]), String(args[0])); case \"List.length\": return int64(len(List(args[0]))); case \"List.rev\": values := List(args[0]); result := make([]any, len(values)); for index, value := range values { result[len(values)-index-1] = value }; return result; case \"List.tryHead\": values := List(args[0]); if len(values) == 0 { return None() }; return Some(values[0]); case \"List.tail\": return Tail(args[0]); case \"List.append\": return Append(List(args[0]), List(args[1])); case \"List.contains\": for _, value := range List(args[1]) { if Equal(args[0], value) { return true } }; return false; case \"Option.isNone\": return !IsSome(args[0]); case \"Option.isSome\": return IsSome(args[0]); case \"Option.defaultValue\": option := AsOption(args[1]); if option.HasValue { return option.Value }; return args[0]; case \"Map.count\": return int64(len(AsMap(args[0]).Entries)); case \"Map.containsKey\": return IsSome(Index(args[1], args[0])); case \"Map.tryGet\": return Index(args[1], args[0]); default: panic(\"Unsupported FScript intrinsic: \" + name) } }\nfunc Format(value any) string { switch item := value.(type) { case bool: if item { return \"true\" }; return \"false\"; case Option: if !item.HasValue { return \"None\" }; return \"Some \" + Format(item.Value); case Union: if !item.HasValue { return item.CaseName }; return item.CaseName + \" \" + Format(item.Value); case OrderedMap: parts := make([]string, len(item.Entries)); for index, entry := range item.Entries { parts[index] = \"[\" + Format(entry.Key) + \"] = \" + Format(entry.Value) }; return \"{ \" + strings.Join(parts, \"; \") + \" }\"; case []any: parts := make([]string, len(item)); for index, value := range item { parts[index] = Format(value) }; return \"[\" + strings.Join(parts, \";\") + \"]\"; default: return fmt.Sprint(value) } }\nfunc Interpolate(values []any) string { var result strings.Builder; for _, value := range values { result.WriteString(Format(value)) }; return result.String() }\nfunc Fail(value any) { panic(fmt.Sprint(value)) }\n" } ] with ex -> Error [ { Code = "FSB4002"; Stage = GoGeneration; Message = ex.Message; Span = None } ] + + let generate options (program: Lir.Program) : Result = + let rec containsTask = function + | TTask _ -> true + | TFun(argument, result) | TMap(argument, result) -> containsTask argument || containsTask result + | TTuple items -> List.exists containsTask items + | TList item | TOption item -> containsTask item + | TRecord fields -> fields |> Map.exists (fun _ -> containsTask) + | TUnion(_, cases) -> cases |> Map.exists (fun _ -> Option.exists containsTask) + | _ -> false + let rec references (expression: Lir.Expression) = + let many values = values |> List.map references |> Set.unionMany + match expression.Node with + | Lir.Load id -> Set.singleton id + | Lir.MakeClosure(_, _, _, body) | Lir.Fail body | Lir.FieldGet(body, _) | Lir.SomeValue body -> references body + | Lir.CallClosure(left, right) | Lir.Append(left, right) | Lir.Cons(left, right) | Lir.Range(left, right) | Lir.IndexGet(left, right) -> Set.union (references left) (references right) + | Lir.Branch(condition, whenTrue, whenFalse) -> many [ condition; whenTrue; whenFalse ] + | Lir.Operation(_, values) | Lir.Tuple values | Lir.List values -> many values + | Lir.Record fields -> fields |> List.map (snd >> references) |> Set.unionMany + | Lir.Let(_, _, _, binding, body) -> Set.union (references binding) (references body) + | Lir.LetRecursiveGroup(bindings, body) -> references body :: (bindings |> List.map (fun (_, _, _, value) -> references value)) |> Set.unionMany + | Lir.DecisionTree(target, cases) -> references target :: (cases |> List.collect (fun case -> [ yield! case.Guard |> Option.map references |> Option.toList; references case.Body ])) |> Set.unionMany + | Lir.RecordUpdate(target, fields) -> references target :: (fields |> List.map (snd >> references)) |> Set.unionMany + | Lir.Interpolate parts -> parts |> List.choose (function Lir.TextPart _ -> None | Lir.ValuePart value -> Some(references value)) |> Set.unionMany + | Lir.UnionCase(_, _, payload) -> payload |> Option.map references |> Option.defaultValue Set.empty + | Lir.Map entries -> entries |> List.map (function Lir.KeyValue(key, value) -> Set.union (references key) (references value) | Lir.Spread value -> references value) |> Set.unionMany + | Lir.ForEach(_, _, _, source, body) -> Set.union (references source) (references body) + | _ -> Set.empty + let declarations = program.Declarations |> List.map (fun declaration -> declaration.Id, declaration) |> Map.ofList + let rec closeReachable visited pending = + match pending with + | [] -> visited + | id :: rest when Set.contains id visited -> closeReachable visited rest + | id :: rest -> + let dependencies = declarations |> Map.tryFind id |> Option.map (_.Value >> references >> Set.toList) |> Option.defaultValue [] + closeReachable (Set.add id visited) (dependencies @ rest) + let reachable = program.Entry |> Option.map (references >> Set.toList >> closeReachable Set.empty) |> Option.defaultValue Set.empty + let taskSpan = + program.Declarations + |> List.tryPick (fun declaration -> if Set.contains declaration.Id reachable && containsTask declaration.Type then Some declaration.Span else None) + |> Option.orElseWith (fun () -> program.Entry |> Option.bind (fun entry -> if containsTask entry.Type then Some entry.Span else None)) + match taskSpan with + | Some span -> Error [ { Code = "FSB4004"; Stage = GoGeneration; Message = "Task values are not supported by the Go backend."; Span = Some span } ] + | None -> generateCore options program diff --git a/src/FScript.Compiler/Compiler.fs b/src/FScript.Compiler/Compiler.fs index a09e107..a4e37f1 100644 --- a/src/FScript.Compiler/Compiler.fs +++ b/src/FScript.Compiler/Compiler.fs @@ -11,15 +11,21 @@ type CompiledProgram = Lir: Lir.Program } module Compiler = + let private descriptor name scheme arity : ExternalFunction = + { Name = name; Scheme = scheme; Arity = arity; Impl = fun _ _ -> VUnit } + let private runtimeContext : ExternalFunction = - { Name = "Env" - Scheme = Forall([], TNamed "Environment") - Arity = 0 - Impl = fun _ _ -> VUnit } + descriptor "Env" (Forall([], TNamed "Environment")) 0 + + let private taskBindings = + [ descriptor "Task.spawn" (Forall([ 0 ], TFun(TFun(TUnit, TVar 0), TTask(TVar 0)))) 1 + descriptor "Task.await" (Forall([ 0 ], TFun(TTask(TVar 0), TVar 0))) 1 ] let compile input = try - let externs = runtimeContext :: (input.Externs |> List.filter (fun external -> external.Name <> runtimeContext.Name)) + let compilerBindings = runtimeContext :: taskBindings + let reserved = compilerBindings |> List.map _.Name |> Set.ofList + let externs = compilerBindings @ (input.Externs |> List.filter (fun external -> not (Set.contains external.Name reserved))) input.Program |> TypeInfer.inferResolvedProgramWithExterns externs |> HirLowering.lower externs diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index da534df..f25e03d 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -9,6 +9,16 @@ open NUnit.Framework [] type GoBackendTests() = + [] + member _.``Go backend reports reachable tasks explicitly`` () = + let compiled = + match Compiler.compile { Program = Parser.parseProgram "Task.await (Task.spawn (fun _ -> 42))"; Externs = [] } with + | Ok value -> value + | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + match GoBackend.generate { ModuleName = "fscript.generated/tasks" } compiled.Lir with + | Ok _ -> Assert.Fail("Expected unsupported task diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSB4004")) + [] [] [] diff --git a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs index 7723e60..e566b17 100644 --- a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs +++ b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs @@ -139,6 +139,23 @@ type LirInterpreterTests() = | Ok _ -> Assert.Fail("Expected unsupported external diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSI3021")) + [] + member _.``Retains tasks in LIR and rejects them only when reached`` () = + let compile source = Compiler.compile { Program = Parser.parseProgram source; Externs = [] } + match compile "let unused = Task.spawn\n1" with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 1L)) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + match compile "Task.await (Task.spawn (fun _ -> 42))" with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + Assert.That(compiled.Lir.Declarations |> List.exists (fun declaration -> match declaration.Value.Node with Lir.HostExternal("Task.spawn", 1) -> true | _ -> false), Is.True) + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok _ -> Assert.Fail("Expected unsupported task diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSI3021")) + [] member _.``Executes local binding patterns iteration and type tokens`` () = let cases = From 4505c2692f3f4c426c91433336bb02a78d5db1a5 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 18:18:37 +0200 Subject: [PATCH 031/133] refactor(compiler): separate extern descriptions --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 ++ docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Compiler/Compiler.fs | 20 +++++++++++-------- src/FScript.Compiler/Diagnostics.fs | 10 ++++++++++ src/FScript.Compiler/HirLowering.fs | 7 +++++-- .../LirInterpreterTests.fs | 4 ++-- 7 files changed, 33 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c4b3f7d..c43f94c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Separated compiler extern descriptors from executable `.NET` external functions and classified runtime-context versus unsupported host linkage. - Preserved `Task.spawn`/`Task.await` as typed host linkage in compiled IR, with reachable-use diagnostics in LIR execution and Go generation. - Added HIR/LIR validation for declaration schemes, source spans, representations, calls, control flow, references, imports, and primitive availability. - Added a serialized-LIR/legacy-evaluator/generated-Go conformance suite and corrected named-record representation in generated functions. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index f47bf70..395bb70 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -66,6 +66,8 @@ Artifact metadata records the actual compiler and language assembly versions, no Compiler bindings classify names as portable intrinsics, backend runtime primitives, target imports, or unsupported host externs. This metadata is distinct from executable .NET `ExternalFunction` implementations. +`CompileInput.Externs` contains `CompilerExtern` descriptors only: name, normalized scheme, arity, and `CompilerExternKind`. Compilation creates private no-op inference shims because the language inference API still consumes `ExternalFunction`, but no executable delegate enters HIR, LIR, artifacts, the interpreter assembly, or a backend assembly. + `Env` is a compiler runtime-context primitive, not a serialized host delegate. LIR interpretation materializes its named record from `InterpreterContext`; generated Go materializes the same fields from `os.Args`, including the executable base name and arguments after it. The LIR interpreter rejects reachable target imports it cannot supply. The Go backend initially rejects unsupported host externs and tasks with backend diagnostics. Imports describe complete ordinary FScript declarations plus target linkage and do not create a second foreign type system. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 9e903e1..5632db6 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -44,6 +44,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Cross-runtime tests compare the legacy evaluator with artifact-round-tripped LIR execution and generated Go for successful values and failure categories. Named records passed through typed function boundaries keep record field projection semantics; their nominal annotation must not select a union runtime representation. - HIR and LIR are accepted only when artifact-local references resolve, source spans are valid, declaration and expression types agree, calls and control-flow nodes have consistent signatures, aggregate representations match their semantic types, and referenced primitives/imports are available and complete. - Task signatures and host linkage remain representable in compiled artifacts. An unused task function handle is inert; fully applying `Task.spawn` or `Task.await` in LIR reports `FSI3021`, while a Go program whose reachable declaration graph requires a task reports `FSB4004`. +- Compiler extern input is declarative: it carries a name, type scheme, arity, and linkage classification, never an executable `.NET` delegate. Host execution values remain confined to the legacy/runtime API. ## Builtins, externs, attributes, and tasks diff --git a/src/FScript.Compiler/Compiler.fs b/src/FScript.Compiler/Compiler.fs index a4e37f1..127a288 100644 --- a/src/FScript.Compiler/Compiler.fs +++ b/src/FScript.Compiler/Compiler.fs @@ -4,30 +4,34 @@ open FScript.Language type CompileInput = { Program: Program - Externs: ExternalFunction list } + Externs: CompilerExtern list } type CompiledProgram = { Hir: Hir.Program Lir: Lir.Program } module Compiler = - let private descriptor name scheme arity : ExternalFunction = - { Name = name; Scheme = scheme; Arity = arity; Impl = fun _ _ -> VUnit } + let private descriptor name scheme arity kind : CompilerExtern = + { Name = name; Scheme = scheme; Arity = arity; Kind = kind } - let private runtimeContext : ExternalFunction = - descriptor "Env" (Forall([], TNamed "Environment")) 0 + let private runtimeContext : CompilerExtern = + descriptor "Env" (Forall([], TNamed "Environment")) 0 RuntimeContextIntrinsic let private taskBindings = - [ descriptor "Task.spawn" (Forall([ 0 ], TFun(TFun(TUnit, TVar 0), TTask(TVar 0)))) 1 - descriptor "Task.await" (Forall([ 0 ], TFun(TTask(TVar 0), TVar 0))) 1 ] + [ descriptor "Task.spawn" (Forall([ 0 ], TFun(TFun(TUnit, TVar 0), TTask(TVar 0)))) 1 UnsupportedHostExtern + descriptor "Task.await" (Forall([ 0 ], TFun(TTask(TVar 0), TVar 0))) 1 UnsupportedHostExtern ] let compile input = try let compilerBindings = runtimeContext :: taskBindings let reserved = compilerBindings |> List.map _.Name |> Set.ofList let externs = compilerBindings @ (input.Externs |> List.filter (fun external -> not (Set.contains external.Name reserved))) + let inferenceExterns = + externs + |> List.map (fun external -> + ({ Name = external.Name; Scheme = external.Scheme; Arity = external.Arity; Impl = fun _ _ -> VUnit }: ExternalFunction)) input.Program - |> TypeInfer.inferResolvedProgramWithExterns externs + |> TypeInfer.inferResolvedProgramWithExterns inferenceExterns |> HirLowering.lower externs |> Result.bind HirValidation.validate |> Result.bind (fun hir -> diff --git a/src/FScript.Compiler/Diagnostics.fs b/src/FScript.Compiler/Diagnostics.fs index 131b676..189252c 100644 --- a/src/FScript.Compiler/Diagnostics.fs +++ b/src/FScript.Compiler/Diagnostics.fs @@ -15,6 +15,16 @@ type CompilerDiagnostic = Message: string Span: Span option } +type CompilerExternKind = + | RuntimeContextIntrinsic + | UnsupportedHostExtern + +type CompilerExtern = + { Name: string + Scheme: Scheme + Arity: int + Kind: CompilerExternKind } + module Diagnostic = let hir code message span = { Code = code diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index 1f060e0..25f91a8 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -317,7 +317,7 @@ module HirLowering = | MESpread value -> lowerExpression state environment inferred value |> Result.map Hir.Spread) |> collectResults |> Result.map (Hir.Map >> make) - let lower (externs: ExternalFunction list) (resolved: TypeInfer.ResolvedProgram) : Result = + let lower (externs: CompilerExtern list) (resolved: TypeInfer.ResolvedProgram) : Result = let expressionTypes = resolved.ExpressionTypes |> List.fold (fun state item -> state |> Map.add item.Expr item.Type) Map.empty @@ -381,7 +381,10 @@ module HirLowering = externs |> List.sortBy _.Name |> List.map (fun external -> let (Forall(_, valueType)) = external.Scheme let span = Span.mk (Span.pos 0 0) (Span.pos 0 0) - let node = if external.Name = "Env" then Hir.RuntimeContext else Hir.HostExternal(external.Name, external.Arity) + let node = + match external.Kind with + | RuntimeContextIntrinsic -> Hir.RuntimeContext + | UnsupportedHostExtern -> Hir.HostExternal(external.Name, external.Arity) ({ Id = externSymbols.[external.Name]; Name = external.Name; Scheme = external.Scheme; IsRecursive = false; IsExported = false; Span = span; Value = { Type = valueType; Span = span; Node = node } }: Hir.Declaration)) builtinDeclarations @ externalDeclarations @ (constructorSymbols |> List.map (fun (typeName, caseName, id) -> let definition, unionType = unionDefinitions |> List.find (fun (definition, _) -> definition.Name = typeName) diff --git a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs index e566b17..b5ad286 100644 --- a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs +++ b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs @@ -120,11 +120,11 @@ type LirInterpreterTests() = [] member _.``Retains host linkage but rejects only reachable external calls`` () = - let external : ExternalFunction = + let external : CompilerExtern = { Name = "Host.echo" Scheme = Forall([], TFun(TString, TString)) Arity = 1 - Impl = fun _ values -> values.Head } + Kind = UnsupportedHostExtern } let compile source = Compiler.compile { Program = Parser.parseProgram source; Externs = [ external ] } match compile "1" with | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) From fbcb1ba646bf7291f5b03a10280ee5828dc00f07 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 18:23:59 +0200 Subject: [PATCH 032/133] feat(compiler): preserve portable console output --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 ++ docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Backend.Go/GoBackend.fs | 7 ++++- src/FScript.Compiler/Compiler.fs | 7 ++++- src/FScript.Compiler/Diagnostics.fs | 6 ++++ src/FScript.Compiler/HirLowering.fs | 1 + src/FScript.Compiler/Lir.fs | 2 +- src/FScript.Interpreter.Lir/LirInterpreter.fs | 20 ++++++++++--- .../GoBackendTests.fs | 1 + .../ConformanceTests.fs | 29 +++++++++++++++---- .../LirInterpreterTests.fs | 11 +++++++ 12 files changed, 76 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c43f94c..7c4cb69 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added portable `Console.writeLine` execution and stdout capture across LIR and generated Go conformance runs. - Separated compiler extern descriptors from executable `.NET` external functions and classified runtime-context versus unsupported host linkage. - Preserved `Task.spawn`/`Task.await` as typed host linkage in compiled IR, with reachable-use diagnostics in LIR execution and Go generation. - Added HIR/LIR validation for declaration schemes, source spans, representations, calls, control flow, references, imports, and primitive availability. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 395bb70..63f26d1 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -68,6 +68,8 @@ Compiler bindings classify names as portable intrinsics, backend runtime primiti `CompileInput.Externs` contains `CompilerExtern` descriptors only: name, normalized scheme, arity, and `CompilerExternKind`. Compilation creates private no-op inference shims because the language inference API still consumes `ExternalFunction`, but no executable delegate enters HIR, LIR, artifacts, the interpreter assembly, or a backend assembly. +`Console.writeLine` is a portable runtime intrinsic. LIR execution captures each line in `ExecutionResult.Stdout` without depending on the process console; generated Go dispatches the same intrinsic to stdout before printing the program result. The cross-runtime harness separates those lines from the final value and compares both channels against the legacy evaluator. + `Env` is a compiler runtime-context primitive, not a serialized host delegate. LIR interpretation materializes its named record from `InterpreterContext`; generated Go materializes the same fields from `os.Args`, including the executable base name and arguments after it. The LIR interpreter rejects reachable target imports it cannot supply. The Go backend initially rejects unsupported host externs and tasks with backend diagnostics. Imports describe complete ordinary FScript declarations plus target linkage and do not create a second foreign type system. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 5632db6..49b6b6e 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -45,6 +45,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - HIR and LIR are accepted only when artifact-local references resolve, source spans are valid, declaration and expression types agree, calls and control-flow nodes have consistent signatures, aggregate representations match their semantic types, and referenced primitives/imports are available and complete. - Task signatures and host linkage remain representable in compiled artifacts. An unused task function handle is inert; fully applying `Task.spawn` or `Task.await` in LIR reports `FSI3021`, while a Go program whose reachable declaration graph requires a task reports `FSB4004`. - Compiler extern input is declarative: it carries a name, type scheme, arity, and linkage classification, never an executable `.NET` delegate. Host execution values remain confined to the legacy/runtime API. +- `Console.writeLine` evaluates its string argument once, appends one stdout line, and returns unit. LIR reports captured lines in order through `ExecutionResult.Stdout`; generated Go emits them in the same order before its final formatted result. ## Builtins, externs, attributes, and tasks diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index c53d2d2..58b472e 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -549,4 +549,9 @@ func Environment() map[string]any { |> Option.orElseWith (fun () -> program.Entry |> Option.bind (fun entry -> if containsTask entry.Type then Some entry.Span else None)) match taskSpan with | Some span -> Error [ { Code = "FSB4004"; Stage = GoGeneration; Message = "Task values are not supported by the Go backend."; Span = Some span } ] - | None -> generateCore options program + | None -> + generateCore options program + |> Result.map (List.map (fun file -> + if file.RelativePath = "internal/fsruntime/runtime.go" then + { file with Content = file.Content.Replace("case \"ignore\":", "case \"Console.writeLine\": fmt.Println(String(args[0])); return struct{}{}; case \"ignore\":") } + else file)) diff --git a/src/FScript.Compiler/Compiler.fs b/src/FScript.Compiler/Compiler.fs index 127a288..525659d 100644 --- a/src/FScript.Compiler/Compiler.fs +++ b/src/FScript.Compiler/Compiler.fs @@ -21,9 +21,14 @@ module Compiler = [ descriptor "Task.spawn" (Forall([ 0 ], TFun(TFun(TUnit, TVar 0), TTask(TVar 0)))) 1 UnsupportedHostExtern descriptor "Task.await" (Forall([ 0 ], TFun(TTask(TVar 0), TVar 0))) 1 UnsupportedHostExtern ] + let private runtimeIntrinsics = + RuntimeIntrinsics.schemes + |> Map.toList + |> List.map (fun (name, scheme) -> descriptor name scheme 1 PortableRuntimeIntrinsic) + let compile input = try - let compilerBindings = runtimeContext :: taskBindings + let compilerBindings = runtimeContext :: (runtimeIntrinsics @ taskBindings) let reserved = compilerBindings |> List.map _.Name |> Set.ofList let externs = compilerBindings @ (input.Externs |> List.filter (fun external -> not (Set.contains external.Name reserved))) let inferenceExterns = diff --git a/src/FScript.Compiler/Diagnostics.fs b/src/FScript.Compiler/Diagnostics.fs index 189252c..12b9f81 100644 --- a/src/FScript.Compiler/Diagnostics.fs +++ b/src/FScript.Compiler/Diagnostics.fs @@ -17,6 +17,7 @@ type CompilerDiagnostic = type CompilerExternKind = | RuntimeContextIntrinsic + | PortableRuntimeIntrinsic | UnsupportedHostExtern type CompilerExtern = @@ -25,6 +26,11 @@ type CompilerExtern = Arity: int Kind: CompilerExternKind } +module RuntimeIntrinsics = + let schemes = + Map.ofList + [ "Console.writeLine", Forall([], TFun(TString, TUnit)) ] + module Diagnostic = let hir code message span = { Code = code diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index 25f91a8..890a3bc 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -384,6 +384,7 @@ module HirLowering = let node = match external.Kind with | RuntimeContextIntrinsic -> Hir.RuntimeContext + | PortableRuntimeIntrinsic -> Hir.Intrinsic external.Name | UnsupportedHostExtern -> Hir.HostExternal(external.Name, external.Arity) ({ Id = externSymbols.[external.Name]; Name = external.Name; Scheme = external.Scheme; IsRecursive = false; IsExported = false; Span = span; Value = { Type = valueType; Span = span; Node = node } }: Hir.Declaration)) builtinDeclarations @ externalDeclarations @ (constructorSymbols |> List.map (fun (typeName, caseName, id) -> diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index bb5c806..615d513 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -316,7 +316,7 @@ module LirValidation = | Lir.Interpolate parts -> parts |> List.iter (function Lir.TextPart _ -> () | Lir.ValuePart value -> check value); if not (compatible expression.Type TString) then report "FSL2004" "LIR interpolation must produce string." expression.Span | Lir.UnionCase(typeName, caseName, payload) -> payload |> Option.iter check; match expression.Type with TUnion(actual, cases) when actual = typeName && Map.containsKey caseName cases -> () | _ -> report "FSL2004" "LIR union representation is inconsistent." expression.Span | Lir.Map entries -> entries |> List.iter (function Lir.KeyValue(key, value) -> check key; check value | Lir.Spread value -> check value); match expression.Type with TMap _ -> () | _ -> report "FSL2004" "LIR map node requires a map type." expression.Span - | Lir.Intrinsic name when not (Map.containsKey name BuiltinSignatures.builtinSchemes) -> report "FSL2009" $"Unknown LIR intrinsic '{name}'." expression.Span + | Lir.Intrinsic name when not (Map.containsKey name BuiltinSignatures.builtinSchemes || Map.containsKey name RuntimeIntrinsics.schemes) -> report "FSL2009" $"Unknown LIR intrinsic '{name}'." expression.Span | Lir.HostExternal(_, arity) when arity < 0 -> report "FSL2010" "LIR host external arity cannot be negative." expression.Span | Lir.TargetImport(modulePath, memberName) when System.String.IsNullOrWhiteSpace(modulePath) || System.String.IsNullOrWhiteSpace(memberName) -> report "FSL2011" "LIR target import linkage is incomplete." expression.Span | Lir.ForEach(id, _, itemType, source, body) -> check source; checkExpression (Set.add id locals) body; match source.Type with TList actual when compatible actual itemType && compatible expression.Type TUnit -> () | _ -> report "FSL2004" "LIR for-loop representation is inconsistent." expression.Span diff --git a/src/FScript.Interpreter.Lir/LirInterpreter.fs b/src/FScript.Interpreter.Lir/LirInterpreter.fs index 4dfa634..51a7776 100644 --- a/src/FScript.Interpreter.Lir/LirInterpreter.fs +++ b/src/FScript.Interpreter.Lir/LirInterpreter.fs @@ -1,5 +1,6 @@ namespace FScript.Compiler +open System.Threading open FScript.Language type InterpreterContext = @@ -25,9 +26,12 @@ type LirValue = | VTargetImport of string * string * int * LirValue list | VClosure of Lir.SymbolId * string * Lir.Expression * Map ref -type ExecutionResult = { Value: LirValue } +type ExecutionResult = + { Value: LirValue + Stdout: string list } module LirInterpreter = + let private currentOutput = AsyncLocal>() let private constantValue = function | Lir.Unit -> VUnit | Lir.Int value -> VInt value | Lir.Float value -> VFloat value | Lir.Bool value -> VBool value | Lir.String value -> VString value @@ -71,7 +75,12 @@ module LirInterpreter = Error [ { Code = code; Stage = Interpreter; Message = message; Span = Some span } ] let private intrinsicArity name = - let (Forall(_, valueType)) = FScript.Language.BuiltinSignatures.builtinSchemes.[name] + let scheme = + FScript.Language.BuiltinSignatures.builtinSchemes + |> Map.tryFind name + |> Option.orElseWith (fun () -> RuntimeIntrinsics.schemes |> Map.tryFind name) + |> Option.defaultWith (fun () -> invalidArg "name" $"Unknown intrinsic '{name}'.") + let (Forall(_, valueType)) = scheme let rec count = function TFun(_, result) -> 1 + count result | _ -> 0 count valueType @@ -93,6 +102,7 @@ module LirInterpreter = | entry :: rest -> loop (entry :: acc) rest loop [] entries match name, arguments with + | "Console.writeLine", [ VString value ] -> currentOutput.Value.Add(value); Ok VUnit | "ignore", [ _ ] -> Ok VUnit | "Int.tryParse", [ VString value ] -> match System.Int64.TryParse value with true, parsed -> Ok(VOption(Some(VInt parsed))) | _ -> Ok(VOption None) | "Float.tryParse", [ VString value ] -> match System.Double.TryParse(value, System.Globalization.NumberStyles.Float, System.Globalization.CultureInfo.InvariantCulture) with true, parsed -> Ok(VOption(Some(VFloat parsed))) | _ -> Ok(VOption None) @@ -438,6 +448,8 @@ module LirInterpreter = match state, result with Ok values, Ok value -> Ok(values @ [ value ]) | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e) (Ok []) let execute (context: InterpreterContext) (program: Lir.Program) = + let output = ResizeArray() + currentOutput.Value <- output let runtimeContext = VRecord(Map.ofList [ "ScriptName", VOption(Option.map VString context.ScriptName) @@ -469,5 +481,5 @@ module LirInterpreter = initialize program.Declarations match failure, program.Entry with | Some errors, _ -> Error errors - | None, Some entry -> evaluate environment.Value entry |> Result.map (fun value -> { Value = value }) - | None, None -> Ok { Value = VUnit } + | None, Some entry -> evaluate environment.Value entry |> Result.map (fun value -> { Value = value; Stdout = List.ofSeq output }) + | None, None -> Ok { Value = VUnit; Stdout = List.ofSeq output } diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index f25e03d..c1d3bed 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -63,6 +63,7 @@ type GoBackendTests() = [] [] [] let upper : string -> string\nupper \"hello\"", "HELLO")>] + [] member _.``Generated projects run with Go`` (source: string, expected: string) = let input = { Program = Parser.parseProgram source; Externs = [] } let compiled = diff --git a/tests/FScript.Conformance.Tests/ConformanceTests.fs b/tests/FScript.Conformance.Tests/ConformanceTests.fs index 99a30c1..b9c2615 100644 --- a/tests/FScript.Conformance.Tests/ConformanceTests.fs +++ b/tests/FScript.Conformance.Tests/ConformanceTests.fs @@ -8,6 +8,8 @@ open FScript.Language open NUnit.Framework module private Harness = + type Outcome = { Value: string; Stdout: string list } + let rec formatLir = function | LirValue.VUnit -> "()" | LirValue.VInt value -> string value @@ -31,13 +33,20 @@ module private Harness = | Ok value -> value | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) - let legacy source = source |> Parser.parseProgram |> TypeInfer.inferProgram |> Eval.evalProgram |> Pretty.valueToString + let legacy source = + let output = ResizeArray() + let console : ExternalFunction = + { Name = "Console.writeLine"; Scheme = Forall([], TFun(TString, TUnit)); Arity = 1 + Impl = fun _ values -> match values with [ VString value ] -> output.Add(value); VUnit | _ -> VUnit } + let program = Parser.parseProgram source + let value = program |> TypeInfer.inferProgramWithExterns [ console ] |> Eval.evalProgramWithExterns [ console ] |> Pretty.valueToString + { Value = value; Stdout = List.ofSeq output } let lir source = let artifact = compile source |> _.Lir |> IrArtifact.serialize let loaded = match IrArtifact.deserialize artifact with Ok value -> value | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) match LirInterpreter.execute { ScriptName = None; Arguments = [] } loaded with - | Ok result -> Ok(formatLir result.Value) + | Ok result -> Ok { Value = formatLir result.Value; Stdout = result.Stdout } | Error diagnostics -> Error diagnostics.Head.Code let go source = @@ -60,7 +69,12 @@ module private Harness = let output = proc.StandardOutput.ReadToEnd().Trim() let error = proc.StandardError.ReadToEnd() proc.WaitForExit() - if proc.ExitCode = 0 then Ok output else Error error + if proc.ExitCode = 0 then + let lines = output.Replace("\r\n", "\n").Split('\n') |> Array.toList + match List.rev lines with + | value :: reversedOutput -> Ok { Value = value; Stdout = List.rev reversedOutput } + | [] -> Ok { Value = ""; Stdout = [] } + else Error error finally if Directory.Exists directory then Directory.Delete(directory, true) @@ -73,13 +87,18 @@ type ConformanceTests() = [ state + value) 0 [1; 2; 3]")>] [] [ value | None -> 0")>] + [] member _.``Legacy LIR artifact and Go results conform`` (source: string) = let expected = Harness.legacy source match Harness.lir source with - | Ok actual -> Assert.That(actual, Is.EqualTo(expected), source) + | Ok actual -> + Assert.That(actual.Value, Is.EqualTo(expected.Value), source) + Assert.That(System.String.Join("\n", actual.Stdout), Is.EqualTo(System.String.Join("\n", expected.Stdout)), source) | Error code -> Assert.Fail($"LIR failed with {code}: {source}") match Harness.go source with - | Ok actual -> Assert.That(actual, Is.EqualTo(expected), source) + | Ok actual -> + Assert.That(actual.Value, Is.EqualTo(expected.Value), source) + Assert.That(System.String.Join("\n", actual.Stdout), Is.EqualTo(System.String.Join("\n", expected.Stdout)), source) | Error error -> Assert.Fail($"Go failed with {error}: {source}") [] diff --git a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs index b5ad286..fed6ae0 100644 --- a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs +++ b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs @@ -6,6 +6,17 @@ open NUnit.Framework [] type LirInterpreterTests() = + [] + member _.``Captures portable console output`` () = + match Compiler.compile { Program = Parser.parseProgram "Console.writeLine \"hello\""; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok result -> + Assert.That(result.Value, Is.EqualTo(LirValue.VUnit)) + Assert.That(System.String.Join("\n", result.Stdout), Is.EqualTo("hello")) + [] member _.``Executes aggregate expressions in LIR`` () = From b7792a2060f7e225ce8bf7c00abbd5b0f1f06cc9 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 18:33:09 +0200 Subject: [PATCH 033/133] feat(go): emit canonical formatted projects --- .github/workflows/ci-main.yml | 5 + .github/workflows/ci-pr.yml | 5 + CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 + docs/specs/compiler-semantics-baseline.md | 1 + .../FScript.Backend.Go.fsproj | 3 + src/FScript.Backend.Go/GoBackend.fs | 93 +++--- src/FScript.Backend.Go/Runtime/closures.go | 193 ++++++++++++ src/FScript.Backend.Go/Runtime/environment.go | 14 + src/FScript.Backend.Go/Runtime/patterns.go | 221 ++++++++++++++ src/FScript.Backend.Go/Runtime/runtime.go | 285 ++++++++++++++++++ .../GoBackendTests.fs | 27 ++ .../ConformanceTests.fs | 14 + 13 files changed, 819 insertions(+), 45 deletions(-) create mode 100644 src/FScript.Backend.Go/Runtime/closures.go create mode 100644 src/FScript.Backend.Go/Runtime/environment.go create mode 100644 src/FScript.Backend.Go/Runtime/patterns.go create mode 100644 src/FScript.Backend.Go/Runtime/runtime.go diff --git a/.github/workflows/ci-main.yml b/.github/workflows/ci-main.yml index 0d78811..be8fe81 100644 --- a/.github/workflows/ci-main.yml +++ b/.github/workflows/ci-main.yml @@ -29,6 +29,11 @@ jobs: with: dotnet-version: 10.0.201 + - name: Setup Go + uses: actions/setup-go@v5 + with: + go-version: '1.25.x' + - name: Restore run: dotnet restore FScript.sln diff --git a/.github/workflows/ci-pr.yml b/.github/workflows/ci-pr.yml index d0ef261..05b7b2f 100644 --- a/.github/workflows/ci-pr.yml +++ b/.github/workflows/ci-pr.yml @@ -35,6 +35,11 @@ jobs: with: dotnet-version: 10.0.201 + - name: Setup Go + uses: actions/setup-go@v5 + with: + go-version: '1.25.x' + - name: Restore run: dotnet restore FScript.sln diff --git a/CHANGELOG.md b/CHANGELOG.md index 7c4cb69..a40d1ba 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Embedded preformatted Go runtime templates and made every generated project pass a zero-diff `gofmt` gate without production toolchain dependence. - Added portable `Console.writeLine` execution and stdout capture across LIR and generated Go conformance runs. - Separated compiler extern descriptors from executable `.NET` external functions and classified runtime-context versus unsupported host linkage. - Preserved `Task.spawn`/`Task.await` as typed host linkage in compiled IR, with reachable-use diagnostics in LIR execution and Go generation. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 63f26d1..3d83519 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -42,6 +42,8 @@ The LIR interpreter implements the pure builtin catalog, including callbacks for Generated Go represents every FScript function as an erased `fsruntime.Closure` with a single `any -> any` call slot. Typed values are checked at closure entry and result use. This uniform convention permits ordinary source closures, partial application, recursion, and higher-order runtime intrinsics to interoperate without Go generics or reflection-based invocation. +The Go runtime is stored as formatted `.go` resources embedded in `FScript.Backend.Go`. Program emission uses multiline structured constructs for closures, local control flow, decision trees, recursive groups, loops, and imports. Generation never invokes Go tooling; the backend tests run `gofmt -d` over every generated project and require an empty diff before `go run`. + Host externs are copied into IR as name, signature, and arity only; their executable .NET delegates never cross the compiler boundary. Constructing or retaining an unused extern handle is inert. Fully applying one in the LIR interpreter produces `FSI3021`, preserving the rule that unreachable unsupported linkage does not block execution. `Task.spawn` and `Task.await` are compiler-known host-linkage descriptors with their polymorphic task signatures. This lets task-typed declarations survive inference, HIR, LIR, and artifact serialization. The LIR interpreter reports `FSI3021` only when either descriptor is fully applied; the Go backend performs deterministic reachability over declaration references and reports `FSB4004` when a reachable declaration requires a task representation. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 49b6b6e..f075310 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -46,6 +46,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Task signatures and host linkage remain representable in compiled artifacts. An unused task function handle is inert; fully applying `Task.spawn` or `Task.await` in LIR reports `FSI3021`, while a Go program whose reachable declaration graph requires a task reports `FSB4004`. - Compiler extern input is declarative: it carries a name, type scheme, arity, and linkage classification, never an executable `.NET` delegate. Host execution values remain confined to the legacy/runtime API. - `Console.writeLine` evaluates its string argument once, appends one stdout line, and returns unit. LIR reports captured lines in order through `ExecutionResult.Stdout`; generated Go emits them in the same order before its final formatted result. +- Generated Go source is deterministic and already canonical according to `gofmt`; generation itself has no installed-Go dependency. Local integration tests skip only when Go is unavailable, while CI installs Go 1.25 and treats its absence as a failure. ## Builtins, externs, attributes, and tasks diff --git a/src/FScript.Backend.Go/FScript.Backend.Go.fsproj b/src/FScript.Backend.Go/FScript.Backend.Go.fsproj index 0f20dbc..5be3974 100644 --- a/src/FScript.Backend.Go/FScript.Backend.Go.fsproj +++ b/src/FScript.Backend.Go/FScript.Backend.Go.fsproj @@ -9,4 +9,7 @@ + + + diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index 58b472e..557ff18 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -1,6 +1,7 @@ namespace FScript.Compiler open System +open System.IO open System.Text open FScript.Language @@ -8,6 +9,19 @@ type GoOptions = { ModuleName: string } type GeneratedFile = { RelativePath: string; Content: string } module GoBackend = + let private indent (value: string) = value.Replace("\n", "\n\t") + + let private runtimeTemplate fileName = + let assembly = typeof.Assembly + let suffix = $".Runtime.{fileName}" + let resourceName = assembly.GetManifestResourceNames() |> Array.find (fun name -> name.EndsWith(suffix, StringComparison.Ordinal)) + use stream = + match assembly.GetManifestResourceStream(resourceName) with + | null -> failwith $"Embedded Go runtime template '{fileName}' was not found." + | value -> value + use reader = new StreamReader(stream) + reader.ReadToEnd() + let private importAlias (modulePath: string) = let hash = Security.Cryptography.SHA256.HashData(Encoding.UTF8.GetBytes(modulePath)) |> Convert.ToHexString "goimport" + hash.Substring(0, 8).ToLowerInvariant() @@ -171,27 +185,15 @@ module GoBackend = let private emitIntrinsic name intrinsicType = let quoted = System.Text.Json.JsonSerializer.Serialize(name) - let rec build index arguments = function - | TFun(_, resultType) -> - let argument = $"intrinsicArg{index}" - let body = build (index + 1) (arguments @ [ argument ]) resultType - $"fsruntime.Closure{{Call: func({argument} any) any {{ return {body} }}}}" - | resultType -> - let values = String.concat ", " arguments - cast resultType $"fsruntime.DispatchIntrinsic({quoted}, []any{{{values}}})" - build 0 [] intrinsicType + let rec arity = function TFun(_, result) -> 1 + arity result | _ -> 0 + let count = arity intrinsicType + if count = 0 then cast intrinsicType $"fsruntime.DispatchIntrinsic({quoted}, nil)" + else $"fsruntime.IntrinsicClosure({quoted}, {count})" let private emitHostExternal name externalType = let quoted = System.Text.Json.JsonSerializer.Serialize(name) - let rec build index arguments = function - | TFun(_, resultType) -> - let argument = $"externalArg{index}" - let body = build (index + 1) (arguments @ [ argument ]) resultType - $"fsruntime.Closure{{Call: func({argument} any) any {{ return {body} }}}}" - | resultType -> - let values = String.concat ", " arguments - cast resultType $"fsruntime.Intrinsic(\"__host:\" + {quoted}, []any{{{values}}})" - build 0 [] externalType + let rec arity = function TFun(_, result) -> 1 + arity result | _ -> 0 + $"fsruntime.HostClosure({quoted}, {arity externalType})" let private emitTargetImport modulePath memberName importType = let alias = importAlias modulePath @@ -200,7 +202,7 @@ module GoBackend = let raw = $"targetArg{index}" let argument = $"targetValue{index}" let body = build (index + 1) (arguments @ [ argument ]) resultType - $"fsruntime.Closure{{Call: func({raw} any) any {{ {argument} := {cast argumentType raw}; return {body} }}}}" + $"fsruntime.Closure{{Call: func({raw} any) any {{\n\t{argument} := {cast argumentType raw}\n\treturn {indent body}\n}}}}" | resultType -> let joined = String.concat ", " arguments match resultType with @@ -219,14 +221,16 @@ module GoBackend = let name = identifier "arg" parameter let parameterValue = cast parameterType "rawValue" emitExpression (environment |> Map.add parameter name) body - |> Result.map (fun bodyCode -> $"fsruntime.Closure{{Call: func(rawValue any) any {{ {name} := {parameterValue}; _ = {name}; return {bodyCode} }}}}") + |> Result.map (fun bodyCode -> + let indentedBody = bodyCode.Replace("\n", "\n\t") + $"fsruntime.Closure{{Call: func(rawValue any) any {{\n\t{name} := {parameterValue}\n\t_ = {name}\n\treturn {indentedBody}\n}}}}") | Lir.CallClosure(fn, argument) -> match emitExpression environment fn, emitExpression environment argument with | Ok fnCode, Ok argumentCode -> Ok(cast expression.Type $"fsruntime.Call({fnCode}, {argumentCode})") | Error error, _ | _, Error error -> Error error | Lir.Branch(condition, whenTrue, whenFalse) -> match emitExpression environment condition, emitExpression environment whenTrue, emitExpression environment whenFalse with - | Ok c, Ok t, Ok f -> Ok $"func() {goType expression.Type} {{ if {c} {{ return {t} }}; return {f} }}()" + | Ok c, Ok t, Ok f -> Ok $"func() {goType expression.Type} {{\n\tif {c} {{\n\t\treturn {indent (indent t)}\n\t}}\n\treturn {indent f}\n}}()" | Error error, _, _ | _, Error error, _ | _, _, Error error -> Error error | Lir.Operation(op, [ left; right ]) -> match emitExpression environment left, emitExpression environment right with @@ -239,7 +243,9 @@ module GoBackend = | Lir.IntAdd | Lir.IntSubtract | Lir.IntMultiply | Lir.IntDivide | Lir.IntRemainder -> $"fsruntime.Int({l})", $"fsruntime.Int({r})" | _ -> l, r - Ok $"({l} {token} {r})" + match op with + | Lir.NumericLessThan | Lir.NumericGreaterThan | Lir.NumericLessThanOrEqual | Lir.NumericGreaterThanOrEqual | Lir.BooleanAnd | Lir.BooleanOr -> Ok $"{l} {token} {r}" + | _ -> Ok $"({l} {token} {r})" | None -> Error $"Unsupported Go operation {op}" | Error error, _ | _, Error error -> Error error | Lir.Tuple items | Lir.List items -> @@ -270,9 +276,9 @@ module GoBackend = match emitExpression scoped binding, emitExpression scoped body with | Ok bindingCode, Ok bodyCode -> let declaration = - if recursive then $"var {name} {goType binding.Type}; {name} = {bindingCode}" - else $"{name} := {bindingCode}" - Ok $"func() {goType expression.Type} {{ {declaration}; return {bodyCode} }}()" + if recursive then $"var {name} {goType binding.Type}\n\t{name} = {indent bindingCode}" + else $"{name} := {indent bindingCode}" + Ok $"func() {goType expression.Type} {{\n\t{declaration}\n\treturn {indent bodyCode}\n}}()" | Error error, _ | _, Error error -> Error error | Lir.LetRecursiveGroup(bindings, body) -> let scoped = bindings |> List.fold (fun env (id, _, _, _) -> Map.add id (identifier "localRec" id) env) environment @@ -282,13 +288,13 @@ module GoBackend = |> collect match loweredBindings, emitExpression scoped body with | Ok values, Ok bodyCode -> - let declarations = values |> List.map (fun (name, bindingType, _) -> $"var {name} {goType bindingType}") |> String.concat "; " - let assignments = values |> List.map (fun (name, _, code) -> $"{name} = {code}") |> String.concat "; " - Ok $"func() {goType expression.Type} {{ {declarations}; {assignments}; return {bodyCode} }}()" + let declarations = values |> List.map (fun (name, bindingType, _) -> $"var {name} {goType bindingType}") |> String.concat "\n\t" + let assignments = values |> List.map (fun (name, _, code) -> $"{name} = {indent code}") |> String.concat "\n\t" + Ok $"func() {goType expression.Type} {{\n\t{declarations}\n\t{assignments}\n\treturn {indent bodyCode}\n}}()" | Error error, _ | _, Error error -> Error error | Lir.Fail value -> emitExpression environment value - |> Result.map (fun code -> $"func() {goType expression.Type} {{ fsruntime.Fail({code}); var zero {goType expression.Type}; return zero }}()") + |> Result.map (fun code -> $"func() {goType expression.Type} {{\n\tfsruntime.Fail({indent code})\n\tvar zero {goType expression.Type}\n\treturn zero\n}}()") | Lir.DecisionTree(target, cases) -> emitExpression environment target |> Result.bind (fun targetCode -> let mutable failure = None @@ -299,13 +305,13 @@ module GoBackend = match case.Guard |> Option.map (emitExpression caseEnvironment) |> Option.defaultValue (Ok "true"), emitExpression caseEnvironment case.Body with | Ok guardCode, Ok bodyCode -> let test, assignments = emitPattern "matchValue" case.Pattern - clauses.Append($"if {test} {{ ") |> ignore - assignments |> List.iter (fun assignment -> clauses.Append(assignment).Append("; ") |> ignore) - clauses.Append($"if {guardCode} {{ return {bodyCode} }} }}; ") |> ignore + clauses.Append($"\tif {test} {{\n") |> ignore + assignments |> List.iter (fun assignment -> clauses.Append("\t\t").Append(assignment.Replace("; ", "\n\t\t")).Append("\n") |> ignore) + clauses.Append($"\t\tif {guardCode} {{\n\t\t\treturn {indent (indent (indent bodyCode))}\n\t\t}}\n\t}}\n") |> ignore | Error error, _ | _, Error error -> failure <- Some error match failure with | Some error -> Error error - | None -> Ok $"func() {goType expression.Type} {{ matchValue := {targetCode}; {clauses}fsruntime.Fail(\"Match failure.\"); var zero {goType expression.Type}; return zero }}()") + | None -> Ok $"func() {goType expression.Type} {{\n\tmatchValue := {indent targetCode}\n{clauses}\tfsruntime.Fail(\"Match failure.\")\n\tvar zero {goType expression.Type}\n\treturn zero\n}}()") | Lir.Range(startValue, endValue) -> match emitExpression environment startValue, emitExpression environment endValue with | Ok first, Ok last -> Ok $"fsruntime.Range({first}, {last})" @@ -354,7 +360,7 @@ module GoBackend = match emitExpression environment source, emitExpression scoped body with | Ok sourceCode, Ok bodyCode -> let itemCode = cast itemType "rawValue" - Ok $"func() struct{{}} {{ for _, rawValue := range fsruntime.List({sourceCode}) {{ {name} := {itemCode}; _ = {bodyCode} }}; return struct{{}}{{}} }}()" + Ok $"func() struct{{}} {{\n\tfor _, rawValue := range fsruntime.List({sourceCode}) {{\n\t\t{name} := {itemCode}\n\t\t_ = {indent (indent bodyCode)}\n\t}}\n\treturn struct{{}}{{}}\n}}()" | Error error, _ | _, Error error -> Error error | Lir.TypeToken tokenType -> Ok(System.Text.Json.JsonSerializer.Serialize(Types.typeToString tokenType)) | Lir.Operation _ -> Error "Invalid operation arity" @@ -472,7 +478,9 @@ func Environment() map[string]any { for declaration in program.Declarations do declarations.AppendLine($"var {names.[declaration.Id]} {goType declaration.Type}") |> ignore match emitExpression names declaration.Value with - | Ok code -> initializers.AppendLine($"\t{names.[declaration.Id]} = {code}") |> ignore + | Ok code -> + let indentedCode = code.Replace("\n", "\n\t") + initializers.AppendLine($"\t{names.[declaration.Id]} = {indentedCode}") |> ignore | Error message -> error <- Some(message, declaration.Span) let entry = match program.Entry with @@ -500,10 +508,10 @@ func Environment() map[string]any { [ { RelativePath = "go.mod"; Content = $"module {moduleName}\n\ngo 1.25\n" } { RelativePath = "main.go"; Content = $"package main\n\nimport (\n\t\"fmt\"\n\t\"{moduleName}/internal/fscriptprogram\"\n\t\"{moduleName}/internal/fsruntime\"\n)\n\nfunc main() {{ {printExpression} }}\n" } { RelativePath = "internal/fscriptprogram/program.go"; Content = programSource } - { RelativePath = "internal/fsruntime/closures.go"; Content = closureRuntime } - { RelativePath = "internal/fsruntime/patterns.go"; Content = patternRuntime } - { RelativePath = "internal/fsruntime/environment.go"; Content = environmentRuntime } - { RelativePath = "internal/fsruntime/runtime.go"; Content = "package fsruntime\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"sort\"\n\t\"strconv\"\n\t\"strings\"\n)\n\ntype Option struct { HasValue bool; Value any }\ntype Union struct { TypeName string; CaseName string; HasValue bool; Value any }\ntype MapEntry struct { Key any; Value any }\ntype OrderedMap struct { Entries []MapEntry }\ntype MapPart struct { IsSpread bool; Key any; Value any; Map OrderedMap }\n\nfunc Unit(value any) struct{} { return struct{}{} }\nfunc Int(value any) int64 { return value.(int64) }\nfunc Float(value any) float64 { return value.(float64) }\nfunc Bool(value any) bool { return value.(bool) }\nfunc String(value any) string { return value.(string) }\nfunc List(value any) []any { return value.([]any) }\nfunc Record(value any) map[string]any { return value.(map[string]any) }\nfunc AsOption(value any) Option { return value.(Option) }\nfunc AsUnion(value any) Union { return value.(Union) }\nfunc AsMap(value any) OrderedMap { return value.(OrderedMap) }\nfunc Some(value any) Option { return Option{HasValue: true, Value: value} }\nfunc None() Option { return Option{} }\nfunc IsSome(value any) bool { return AsOption(value).HasValue }\nfunc OptionValue(value any) any { return AsOption(value).Value }\nfunc MakeUnion(typeName string, caseName string) Union { return Union{TypeName: typeName, CaseName: caseName} }\nfunc MakeUnionValue(typeName string, caseName string, value any) Union { return Union{TypeName: typeName, CaseName: caseName, HasValue: true, Value: value} }\nfunc IsUnion(value any, typeName string, caseName string) bool { item := AsUnion(value); return item.TypeName == typeName && item.CaseName == caseName }\nfunc UnionHasValue(value any) bool { return AsUnion(value).HasValue }\nfunc UnionValue(value any) any { return AsUnion(value).Value }\nfunc HasField(value any, name string) bool { _, ok := Record(value)[name]; return ok }\nfunc Equal(left any, right any) bool { return reflect.DeepEqual(left, right) }\nfunc Cons(head any, tail []any) []any { return append([]any{head}, tail...) }\nfunc Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) }\nfunc Tail(value any) []any { return append([]any{}, List(value)[1:]...) }\nfunc Range(first int64, last int64) []any { step := int64(1); if first > last { step = -1 }; values := []any{}; for current := first; ; current += step { values = append(values, current); if current == last { break } }; return values }\nfunc UpdateRecord(value any, updates map[string]any) map[string]any { result := map[string]any{}; for key, item := range Record(value) { result[key] = item }; for key, item := range updates { result[key] = item }; return result }\nfunc Pair(key any, value any) MapPart { return MapPart{Key: key, Value: value} }\nfunc SpreadMap(value any) MapPart { return MapPart{IsSpread: true, Map: AsMap(value)} }\nfunc mapSet(entries []MapEntry, key any, value any, replace bool) []MapEntry { for index, entry := range entries { if Equal(entry.Key, key) { if replace { entries[index].Value = value }; return entries } }; return append(entries, MapEntry{Key: key, Value: value}) }\nfunc BuildMap(parts []MapPart) OrderedMap { entries := []MapEntry{}; for _, part := range parts { if part.IsSpread { for _, entry := range part.Map.Entries { entries = mapSet(entries, entry.Key, entry.Value, false) } } else { entries = mapSet(entries, part.Key, part.Value, true) } }; sort.SliceStable(entries, func(left int, right int) bool { return Format(entries[left].Key) < Format(entries[right].Key) }); return OrderedMap{Entries: entries} }\nfunc Index(value any, index any) Option { switch target := value.(type) { case []any: position := Int(index); if position < 0 || position >= int64(len(target)) { return None() }; return Some(target[position]); case OrderedMap: for _, entry := range target.Entries { if Equal(entry.Key, index) { return Some(entry.Value) } }; return None(); default: panic(\"Index access requires a list or map\") } }\nfunc Intrinsic(name string, args []any) any { switch name { case \"ignore\": return struct{}{}; case \"List.empty\": return []any{}; case \"Map.empty\": return OrderedMap{}; case \"Int.tryParse\": value, error := strconv.ParseInt(String(args[0]), 10, 64); if error != nil { return None() }; return Some(value); case \"Float.tryParse\": value, error := strconv.ParseFloat(String(args[0]), 64); if error != nil { return None() }; return Some(value); case \"Bool.tryParse\": value, error := strconv.ParseBool(String(args[0])); if error != nil { return None() }; return Some(value); case \"Int.toString\", \"Float.toString\", \"Bool.toString\": return Format(args[0]); case \"String.replace\": return strings.ReplaceAll(String(args[2]), String(args[0]), String(args[1])); case \"String.indexOf\": index := strings.Index(String(args[1]), String(args[0])); if index < 0 { return None() }; return Some(int64(index)); case \"String.toLower\": return strings.ToLower(String(args[0])); case \"String.toUpper\": return strings.ToUpper(String(args[0])); case \"String.substring\": start, length, value := Int(args[0]), Int(args[1]), String(args[2]); if start < 0 || length < 0 || start + length > int64(len(value)) { return None() }; return Some(value[start:start+length]); case \"String.concat\": values := List(args[1]); parts := make([]string, len(values)); for index, value := range values { parts[index] = String(value) }; return strings.Join(parts, String(args[0])); case \"String.split\": values := strings.Split(String(args[1]), String(args[0])); result := make([]any, len(values)); for index, value := range values { result[index] = value }; return result; case \"String.endsWith\": return strings.HasSuffix(String(args[1]), String(args[0])); case \"List.length\": return int64(len(List(args[0]))); case \"List.rev\": values := List(args[0]); result := make([]any, len(values)); for index, value := range values { result[len(values)-index-1] = value }; return result; case \"List.tryHead\": values := List(args[0]); if len(values) == 0 { return None() }; return Some(values[0]); case \"List.tail\": return Tail(args[0]); case \"List.append\": return Append(List(args[0]), List(args[1])); case \"List.contains\": for _, value := range List(args[1]) { if Equal(args[0], value) { return true } }; return false; case \"Option.isNone\": return !IsSome(args[0]); case \"Option.isSome\": return IsSome(args[0]); case \"Option.defaultValue\": option := AsOption(args[1]); if option.HasValue { return option.Value }; return args[0]; case \"Map.count\": return int64(len(AsMap(args[0]).Entries)); case \"Map.containsKey\": return IsSome(Index(args[1], args[0])); case \"Map.tryGet\": return Index(args[1], args[0]); default: panic(\"Unsupported FScript intrinsic: \" + name) } }\nfunc Format(value any) string { switch item := value.(type) { case bool: if item { return \"true\" }; return \"false\"; case Option: if !item.HasValue { return \"None\" }; return \"Some \" + Format(item.Value); case Union: if !item.HasValue { return item.CaseName }; return item.CaseName + \" \" + Format(item.Value); case OrderedMap: parts := make([]string, len(item.Entries)); for index, entry := range item.Entries { parts[index] = \"[\" + Format(entry.Key) + \"] = \" + Format(entry.Value) }; return \"{ \" + strings.Join(parts, \"; \") + \" }\"; case []any: parts := make([]string, len(item)); for index, value := range item { parts[index] = Format(value) }; return \"[\" + strings.Join(parts, \";\") + \"]\"; default: return fmt.Sprint(value) } }\nfunc Interpolate(values []any) string { var result strings.Builder; for _, value := range values { result.WriteString(Format(value)) }; return result.String() }\nfunc Fail(value any) { panic(fmt.Sprint(value)) }\n" } ] + { RelativePath = "internal/fsruntime/closures.go"; Content = runtimeTemplate "closures.go" } + { RelativePath = "internal/fsruntime/patterns.go"; Content = runtimeTemplate "patterns.go" } + { RelativePath = "internal/fsruntime/environment.go"; Content = runtimeTemplate "environment.go" } + { RelativePath = "internal/fsruntime/runtime.go"; Content = runtimeTemplate "runtime.go" } ] with ex -> Error [ { Code = "FSB4002"; Stage = GoGeneration; Message = ex.Message; Span = None } ] @@ -549,9 +557,4 @@ func Environment() map[string]any { |> Option.orElseWith (fun () -> program.Entry |> Option.bind (fun entry -> if containsTask entry.Type then Some entry.Span else None)) match taskSpan with | Some span -> Error [ { Code = "FSB4004"; Stage = GoGeneration; Message = "Task values are not supported by the Go backend."; Span = Some span } ] - | None -> - generateCore options program - |> Result.map (List.map (fun file -> - if file.RelativePath = "internal/fsruntime/runtime.go" then - { file with Content = file.Content.Replace("case \"ignore\":", "case \"Console.writeLine\": fmt.Println(String(args[0])); return struct{}{}; case \"ignore\":") } - else file)) + | None -> generateCore options program diff --git a/src/FScript.Backend.Go/Runtime/closures.go b/src/FScript.Backend.Go/Runtime/closures.go new file mode 100644 index 0000000..b605373 --- /dev/null +++ b/src/FScript.Backend.Go/Runtime/closures.go @@ -0,0 +1,193 @@ +package fsruntime + +type Closure struct{ Call func(any) any } + +func AsClosure(value any) Closure { return value.(Closure) } +func Call(function any, argument any) any { return AsClosure(function).Call(argument) } + +func IntrinsicClosure(name string, arity int) Closure { + var build func([]any) Closure + build = func(arguments []any) Closure { + return Closure{Call: func(value any) any { + next := append(append([]any{}, arguments...), value) + if len(next) == arity { + return DispatchIntrinsic(name, next) + } + return build(next) + }} + } + return build(nil) +} + +func HostClosure(name string, arity int) Closure { + var build func(int) Closure + build = func(count int) Closure { + return Closure{Call: func(value any) any { + _ = value + if count+1 == arity { + panic("Unsupported host external: " + name) + } + return build(count + 1) + }} + } + return build(0) +} + +func DispatchIntrinsic(name string, args []any) any { + switch name { + case "List.map": + callback, values := AsClosure(args[0]), List(args[1]) + result := make([]any, len(values)) + for index, value := range values { + result[index] = callback.Call(value) + } + return result + case "List.iter": + callback := AsClosure(args[0]) + for _, value := range List(args[1]) { + callback.Call(value) + } + return struct{}{} + case "List.choose": + callback, result := AsClosure(args[0]), []any{} + for _, value := range List(args[1]) { + chosen := AsOption(callback.Call(value)) + if chosen.HasValue { + result = append(result, chosen.Value) + } + } + return result + case "List.collect": + callback, result := AsClosure(args[0]), []any{} + for _, value := range List(args[1]) { + result = append(result, List(callback.Call(value))...) + } + return result + case "List.exists": + callback := AsClosure(args[0]) + for _, value := range List(args[1]) { + if Bool(callback.Call(value)) { + return true + } + } + return false + case "List.distinct": + result := []any{} + for _, value := range List(args[0]) { + found := false + for _, existing := range result { + if Equal(existing, value) { + found = true + break + } + } + if !found { + result = append(result, value) + } + } + return result + case "List.fold": + callback, state := AsClosure(args[0]), args[1] + for _, value := range List(args[2]) { + state = Call(callback.Call(state), value) + } + return state + case "List.filter": + callback, result := AsClosure(args[0]), []any{} + for _, value := range List(args[1]) { + if Bool(callback.Call(value)) { + result = append(result, value) + } + } + return result + case "List.tryFind": + callback := AsClosure(args[0]) + for _, value := range List(args[1]) { + if Bool(callback.Call(value)) { + return Some(value) + } + } + return None() + case "List.tryGet": + callback := AsClosure(args[0]) + for index, value := range List(args[1]) { + if Bool(callback.Call(value)) { + return Some(int64(index)) + } + } + return None() + case "List.tryItem": + index, values := Int(args[0]), List(args[1]) + if index < 0 || index >= int64(len(values)) { + return None() + } + return Some(values[index]) + case "Option.defaultWith": + option := AsOption(args[1]) + if option.HasValue { + return option.Value + } + return Call(args[0], struct{}{}) + case "Option.map": + option := AsOption(args[1]) + if !option.HasValue { + return None() + } + return Some(Call(args[0], option.Value)) + case "Map.add": + values := append([]MapEntry{}, AsMap(args[2]).Entries...) + return OrderedMap{Entries: mapSet(values, args[0], args[1], true)} + case "Map.ofList": + values := []MapEntry{} + for _, pair := range List(args[0]) { + tuple := List(pair) + values = mapSet(values, tuple[0], tuple[1], true) + } + return OrderedMap{Entries: values} + case "Map.fold": + callback, state := AsClosure(args[0]), args[1] + for _, entry := range AsMap(args[2]).Entries { + state = Call(Call(callback.Call(state), entry.Key), entry.Value) + } + return state + case "Map.filter": + callback, result := AsClosure(args[0]), []MapEntry{} + for _, entry := range AsMap(args[1]).Entries { + if Bool(Call(callback.Call(entry.Key), entry.Value)) { + result = append(result, entry) + } + } + return OrderedMap{Entries: result} + case "Map.choose": + callback, result := AsClosure(args[0]), []MapEntry{} + for _, entry := range AsMap(args[1]).Entries { + chosen := AsOption(Call(callback.Call(entry.Key), entry.Value)) + if chosen.HasValue { + result = append(result, MapEntry{Key: entry.Key, Value: chosen.Value}) + } + } + return OrderedMap{Entries: result} + case "Map.map": + callback, result := AsClosure(args[0]), []MapEntry{} + for _, entry := range AsMap(args[1]).Entries { + result = append(result, MapEntry{Key: entry.Key, Value: callback.Call(entry.Value)}) + } + return OrderedMap{Entries: result} + case "Map.iter": + callback := AsClosure(args[0]) + for _, entry := range AsMap(args[1]).Entries { + Call(callback.Call(entry.Key), entry.Value) + } + return struct{}{} + case "Map.remove": + result := []MapEntry{} + for _, entry := range AsMap(args[1]).Entries { + if !Equal(entry.Key, args[0]) { + result = append(result, entry) + } + } + return OrderedMap{Entries: result} + default: + return Intrinsic(name, args) + } +} diff --git a/src/FScript.Backend.Go/Runtime/environment.go b/src/FScript.Backend.Go/Runtime/environment.go new file mode 100644 index 0000000..ce59859 --- /dev/null +++ b/src/FScript.Backend.Go/Runtime/environment.go @@ -0,0 +1,14 @@ +package fsruntime + +import ( + "os" + "path/filepath" +) + +func Environment() map[string]any { + arguments := make([]any, len(os.Args)-1) + for index, argument := range os.Args[1:] { + arguments[index] = argument + } + return map[string]any{"ScriptName": Some(filepath.Base(os.Args[0])), "Arguments": arguments} +} diff --git a/src/FScript.Backend.Go/Runtime/patterns.go b/src/FScript.Backend.Go/Runtime/patterns.go new file mode 100644 index 0000000..a4f5179 --- /dev/null +++ b/src/FScript.Backend.Go/Runtime/patterns.go @@ -0,0 +1,221 @@ +package fsruntime + +import "strings" + +type Pattern struct { + Kind string + Symbol int + Literal any + Items []Pattern + Fields []FieldPattern + Clauses []MapPatternClause + Tail *Pattern + TypeName string + CaseName string +} +type FieldPattern struct { + Name string + Pattern Pattern +} +type MapPatternClause struct { + Key Pattern + Value Pattern +} + +func PatternPtr(value Pattern) *Pattern { return &value } + +func mergePatternBindings(left map[int]any, right map[int]any) (map[int]any, bool) { + result := map[int]any{} + for key, value := range left { + result[key] = value + } + for key, value := range right { + if existing, found := result[key]; found && !Equal(existing, value) { + return nil, false + } + result[key] = value + } + return result, true +} + +func matchRuntimePattern(value any, pattern Pattern) (map[int]any, bool) { + switch pattern.Kind { + case "wildcard": + return map[int]any{}, true + case "binding": + return map[int]any{pattern.Symbol: value}, true + case "literal": + return map[int]any{}, Equal(value, pattern.Literal) + case "empty-list": + values, ok := value.([]any) + return map[int]any{}, ok && len(values) == 0 + case "none": + option, ok := value.(Option) + return map[int]any{}, ok && !option.HasValue + case "some": + option, ok := value.(Option) + if !ok || !option.HasValue { + return nil, false + } + return matchRuntimePattern(option.Value, pattern.Items[0]) + case "cons": + values, ok := value.([]any) + if !ok || len(values) == 0 { + return nil, false + } + left, matched := matchRuntimePattern(values[0], pattern.Items[0]) + if !matched { + return nil, false + } + right, matched := matchRuntimePattern(values[1:], pattern.Items[1]) + if !matched { + return nil, false + } + return mergePatternBindings(left, right) + case "tuple": + values, ok := value.([]any) + if !ok || len(values) != len(pattern.Items) { + return nil, false + } + bindings := map[int]any{} + for index, item := range pattern.Items { + next, matched := matchRuntimePattern(values[index], item) + if !matched { + return nil, false + } + bindings, matched = mergePatternBindings(bindings, next) + if !matched { + return nil, false + } + } + return bindings, true + case "record": + values, ok := value.(map[string]any) + if !ok { + return nil, false + } + bindings := map[int]any{} + for _, field := range pattern.Fields { + item, found := values[field.Name] + if !found { + return nil, false + } + next, matched := matchRuntimePattern(item, field.Pattern) + if !matched { + return nil, false + } + bindings, matched = mergePatternBindings(bindings, next) + if !matched { + return nil, false + } + } + return bindings, true + case "union": + item, ok := value.(Union) + if !ok || item.TypeName != pattern.TypeName || item.CaseName != pattern.CaseName || (len(pattern.Items) == 0) != !item.HasValue { + return nil, false + } + if len(pattern.Items) == 0 { + return map[int]any{}, true + } + return matchRuntimePattern(item.Value, pattern.Items[0]) + case "type": + return map[int]any{}, TypeMatches(value, pattern.TypeName) + case "map": + item, ok := value.(OrderedMap) + if !ok { + return nil, false + } + remaining := append([]MapEntry{}, item.Entries...) + bindings := map[int]any{} + for _, clause := range pattern.Clauses { + found := false + for index, entry := range remaining { + keyBindings, keyMatched := matchRuntimePattern(entry.Key, clause.Key) + if !keyMatched { + continue + } + withKey, merged := mergePatternBindings(bindings, keyBindings) + if !merged { + continue + } + valueBindings, valueMatched := matchRuntimePattern(entry.Value, clause.Value) + if !valueMatched { + continue + } + withValue, merged := mergePatternBindings(withKey, valueBindings) + if !merged { + continue + } + bindings = withValue + remaining = append(remaining[:index], remaining[index+1:]...) + found = true + break + } + if !found { + return nil, false + } + } + if pattern.Tail != nil { + tailBindings, matched := matchRuntimePattern(OrderedMap{Entries: remaining}, *pattern.Tail) + if !matched { + return nil, false + } + return mergePatternBindings(bindings, tailBindings) + } + if len(pattern.Clauses) > 0 || len(remaining) == 0 { + return bindings, true + } + return nil, false + default: + return nil, false + } +} + +func PatternMatches(value any, pattern Pattern) bool { + _, matched := matchRuntimePattern(value, pattern) + return matched +} +func PatternBinding(value any, pattern Pattern, symbol int) any { + bindings, matched := matchRuntimePattern(value, pattern) + if !matched { + panic("Pattern did not match") + } + return bindings[symbol] +} +func TypeMatches(value any, typeName string) bool { + switch typeName { + case "unit": + _, ok := value.(struct{}) + return ok + case "int": + _, ok := value.(int64) + return ok + case "float": + _, ok := value.(float64) + return ok + case "bool": + _, ok := value.(bool) + return ok + case "string": + _, ok := value.(string) + return ok + } + if strings.HasSuffix(typeName, " list") { + _, ok := value.([]any) + return ok + } + if strings.HasSuffix(typeName, " option") { + _, ok := value.(Option) + return ok + } + if strings.HasPrefix(typeName, "{") { + _, ok := value.(map[string]any) + return ok + } + if item, ok := value.(Union); ok { + return item.TypeName == typeName + } + _, record := value.(map[string]any) + return record +} diff --git a/src/FScript.Backend.Go/Runtime/runtime.go b/src/FScript.Backend.Go/Runtime/runtime.go new file mode 100644 index 0000000..7461454 --- /dev/null +++ b/src/FScript.Backend.Go/Runtime/runtime.go @@ -0,0 +1,285 @@ +package fsruntime + +import ( + "fmt" + "reflect" + "sort" + "strconv" + "strings" +) + +type Option struct { + HasValue bool + Value any +} +type Union struct { + TypeName string + CaseName string + HasValue bool + Value any +} +type MapEntry struct { + Key any + Value any +} +type OrderedMap struct{ Entries []MapEntry } +type MapPart struct { + IsSpread bool + Key any + Value any + Map OrderedMap +} + +func Unit(value any) struct{} { return struct{}{} } +func Int(value any) int64 { return value.(int64) } +func Float(value any) float64 { return value.(float64) } +func Bool(value any) bool { return value.(bool) } +func String(value any) string { return value.(string) } +func List(value any) []any { return value.([]any) } +func Record(value any) map[string]any { return value.(map[string]any) } +func AsOption(value any) Option { return value.(Option) } +func AsUnion(value any) Union { return value.(Union) } +func AsMap(value any) OrderedMap { return value.(OrderedMap) } +func Some(value any) Option { return Option{HasValue: true, Value: value} } +func None() Option { return Option{} } +func IsSome(value any) bool { return AsOption(value).HasValue } +func OptionValue(value any) any { return AsOption(value).Value } +func MakeUnion(typeName string, caseName string) Union { + return Union{TypeName: typeName, CaseName: caseName} +} +func MakeUnionValue(typeName string, caseName string, value any) Union { + return Union{TypeName: typeName, CaseName: caseName, HasValue: true, Value: value} +} +func IsUnion(value any, typeName string, caseName string) bool { + item := AsUnion(value) + return item.TypeName == typeName && item.CaseName == caseName +} +func UnionHasValue(value any) bool { return AsUnion(value).HasValue } +func UnionValue(value any) any { return AsUnion(value).Value } +func HasField(value any, name string) bool { _, ok := Record(value)[name]; return ok } +func Equal(left any, right any) bool { return reflect.DeepEqual(left, right) } +func Cons(head any, tail []any) []any { return append([]any{head}, tail...) } +func Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) } +func Tail(value any) []any { return append([]any{}, List(value)[1:]...) } +func Range(first int64, last int64) []any { + step := int64(1) + if first > last { + step = -1 + } + values := []any{} + for current := first; ; current += step { + values = append(values, current) + if current == last { + break + } + } + return values +} +func UpdateRecord(value any, updates map[string]any) map[string]any { + result := map[string]any{} + for key, item := range Record(value) { + result[key] = item + } + for key, item := range updates { + result[key] = item + } + return result +} +func Pair(key any, value any) MapPart { return MapPart{Key: key, Value: value} } +func SpreadMap(value any) MapPart { return MapPart{IsSpread: true, Map: AsMap(value)} } +func mapSet(entries []MapEntry, key any, value any, replace bool) []MapEntry { + for index, entry := range entries { + if Equal(entry.Key, key) { + if replace { + entries[index].Value = value + } + return entries + } + } + return append(entries, MapEntry{Key: key, Value: value}) +} +func BuildMap(parts []MapPart) OrderedMap { + entries := []MapEntry{} + for _, part := range parts { + if part.IsSpread { + for _, entry := range part.Map.Entries { + entries = mapSet(entries, entry.Key, entry.Value, false) + } + } else { + entries = mapSet(entries, part.Key, part.Value, true) + } + } + sort.SliceStable(entries, func(left int, right int) bool { return Format(entries[left].Key) < Format(entries[right].Key) }) + return OrderedMap{Entries: entries} +} +func Index(value any, index any) Option { + switch target := value.(type) { + case []any: + position := Int(index) + if position < 0 || position >= int64(len(target)) { + return None() + } + return Some(target[position]) + case OrderedMap: + for _, entry := range target.Entries { + if Equal(entry.Key, index) { + return Some(entry.Value) + } + } + return None() + default: + panic("Index access requires a list or map") + } +} +func Intrinsic(name string, args []any) any { + switch name { + case "Console.writeLine": + fmt.Println(String(args[0])) + return struct{}{} + case "ignore": + return struct{}{} + case "List.empty": + return []any{} + case "Map.empty": + return OrderedMap{} + case "Int.tryParse": + value, error := strconv.ParseInt(String(args[0]), 10, 64) + if error != nil { + return None() + } + return Some(value) + case "Float.tryParse": + value, error := strconv.ParseFloat(String(args[0]), 64) + if error != nil { + return None() + } + return Some(value) + case "Bool.tryParse": + value, error := strconv.ParseBool(String(args[0])) + if error != nil { + return None() + } + return Some(value) + case "Int.toString", "Float.toString", "Bool.toString": + return Format(args[0]) + case "String.replace": + return strings.ReplaceAll(String(args[2]), String(args[0]), String(args[1])) + case "String.indexOf": + index := strings.Index(String(args[1]), String(args[0])) + if index < 0 { + return None() + } + return Some(int64(index)) + case "String.toLower": + return strings.ToLower(String(args[0])) + case "String.toUpper": + return strings.ToUpper(String(args[0])) + case "String.substring": + start, length, value := Int(args[0]), Int(args[1]), String(args[2]) + if start < 0 || length < 0 || start+length > int64(len(value)) { + return None() + } + return Some(value[start : start+length]) + case "String.concat": + values := List(args[1]) + parts := make([]string, len(values)) + for index, value := range values { + parts[index] = String(value) + } + return strings.Join(parts, String(args[0])) + case "String.split": + values := strings.Split(String(args[1]), String(args[0])) + result := make([]any, len(values)) + for index, value := range values { + result[index] = value + } + return result + case "String.endsWith": + return strings.HasSuffix(String(args[1]), String(args[0])) + case "List.length": + return int64(len(List(args[0]))) + case "List.rev": + values := List(args[0]) + result := make([]any, len(values)) + for index, value := range values { + result[len(values)-index-1] = value + } + return result + case "List.tryHead": + values := List(args[0]) + if len(values) == 0 { + return None() + } + return Some(values[0]) + case "List.tail": + return Tail(args[0]) + case "List.append": + return Append(List(args[0]), List(args[1])) + case "List.contains": + for _, value := range List(args[1]) { + if Equal(args[0], value) { + return true + } + } + return false + case "Option.isNone": + return !IsSome(args[0]) + case "Option.isSome": + return IsSome(args[0]) + case "Option.defaultValue": + option := AsOption(args[1]) + if option.HasValue { + return option.Value + } + return args[0] + case "Map.count": + return int64(len(AsMap(args[0]).Entries)) + case "Map.containsKey": + return IsSome(Index(args[1], args[0])) + case "Map.tryGet": + return Index(args[1], args[0]) + default: + panic("Unsupported FScript intrinsic: " + name) + } +} +func Format(value any) string { + switch item := value.(type) { + case bool: + if item { + return "true" + } + return "false" + case Option: + if !item.HasValue { + return "None" + } + return "Some " + Format(item.Value) + case Union: + if !item.HasValue { + return item.CaseName + } + return item.CaseName + " " + Format(item.Value) + case OrderedMap: + parts := make([]string, len(item.Entries)) + for index, entry := range item.Entries { + parts[index] = "[" + Format(entry.Key) + "] = " + Format(entry.Value) + } + return "{ " + strings.Join(parts, "; ") + " }" + case []any: + parts := make([]string, len(item)) + for index, value := range item { + parts[index] = Format(value) + } + return "[" + strings.Join(parts, ";") + "]" + default: + return fmt.Sprint(value) + } +} +func Interpolate(values []any) string { + var result strings.Builder + for _, value := range values { + result.WriteString(Format(value)) + } + return result.String() +} +func Fail(value any) { panic(fmt.Sprint(value)) } diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index c1d3bed..ff49a0c 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -9,6 +9,19 @@ open NUnit.Framework [] type GoBackendTests() = + let requireGo () = + try + use proc = + match Process.Start(ProcessStartInfo("go", "version", RedirectStandardOutput = true)) with + | null -> raise (InvalidOperationException("Could not start go")) + | value -> value + proc.WaitForExit() + if proc.ExitCode <> 0 then raise (InvalidOperationException("go version failed")) + with ex -> + if String.Equals(Environment.GetEnvironmentVariable("CI"), "true", StringComparison.OrdinalIgnoreCase) then + Assert.Fail($"Go is required in CI: {ex.Message}") + else Assert.Ignore($"Go is not installed: {ex.Message}") + [] member _.``Go backend reports reachable tasks explicitly`` () = let compiled = @@ -65,6 +78,7 @@ type GoBackendTests() = [] let upper : string -> string\nupper \"hello\"", "HELLO")>] [] member _.``Generated projects run with Go`` (source: string, expected: string) = + requireGo () let input = { Program = Parser.parseProgram source; Externs = [] } let compiled = match Compiler.compile input with @@ -82,6 +96,19 @@ type GoBackendTests() = | null -> () | parent -> Directory.CreateDirectory(parent) |> ignore File.WriteAllText(path, file.Content) + let formatStart = ProcessStartInfo("gofmt") + formatStart.WorkingDirectory <- directory + formatStart.ArgumentList.Add("-d") + formatStart.ArgumentList.Add(directory) + formatStart.RedirectStandardOutput <- true + use formatProcess = + match Process.Start(formatStart) with + | null -> failwith "Failed to start gofmt" + | value -> value + let formatDiff = formatProcess.StandardOutput.ReadToEnd() + formatProcess.WaitForExit() + Assert.That(formatProcess.ExitCode, Is.EqualTo(0)) + Assert.That(formatDiff, Is.Empty, "Generated Go sources must already be gofmt-compatible") let start = ProcessStartInfo("go") start.WorkingDirectory <- directory start.ArgumentList.Add("run") diff --git a/tests/FScript.Conformance.Tests/ConformanceTests.fs b/tests/FScript.Conformance.Tests/ConformanceTests.fs index b9c2615..80c686c 100644 --- a/tests/FScript.Conformance.Tests/ConformanceTests.fs +++ b/tests/FScript.Conformance.Tests/ConformanceTests.fs @@ -80,6 +80,20 @@ module private Harness = [] type ConformanceTests() = + [] + member _.``Require Go toolchain`` () = + try + use proc = + match Process.Start(ProcessStartInfo("go", "version", RedirectStandardOutput = true)) with + | null -> raise (InvalidOperationException("Could not start go")) + | value -> value + proc.WaitForExit() + if proc.ExitCode <> 0 then raise (InvalidOperationException("go version failed")) + with ex -> + if String.Equals(Environment.GetEnvironmentVariable("CI"), "true", StringComparison.OrdinalIgnoreCase) then + Assert.Fail($"Go is required in CI: {ex.Message}") + else Assert.Ignore($"Go is not installed: {ex.Message}") + [] [] [ 0 | Circle radius -> radius")>] From 00f1c8680f395b63c67a0fe6f1b4f4844e0484da Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 18:34:19 +0200 Subject: [PATCH 034/133] feat(compiler): expose LIR optimization stage --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 ++ src/FScript.Compiler/Compiler.fs | 2 +- src/FScript.Compiler/Lir.fs | 5 +++++ tests/FScript.Compiler.Tests/HirTests.fs | 6 ++++++ 5 files changed, 15 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a40d1ba..e428a1d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added an explicit identity LIR optimization pass between lowering and validation for future semantic transformations. - Embedded preformatted Go runtime templates and made every generated project pass a zero-diff `gofmt` gate without production toolchain dependence. - Added portable `Console.writeLine` execution and stdout capture across LIR and generated Go conformance runs. - Separated compiler extern descriptors from executable `.NET` external functions and classified runtime-context versus unsupported host linkage. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 3d83519..a3cecf4 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -24,6 +24,8 @@ HIR is language-oriented and contains resolved artifact-local symbol identities, LIR is executable but not a Go AST. It uses lifted functions, explicit captures and closure creation, resolved primitive operations, aggregate construction/projection, and structured control flow. Pattern matching becomes deterministic tests and projections before reaching a backend. Semantic types may remain alongside selected runtime representations for validation and diagnostics. +The pass pipeline is HIR lowering, HIR validation, LIR lowering, `LirOptimization.optimize`, then LIR validation. The optimization pass is intentionally identity-only in version 1; its explicit boundary prevents later transformations from being hidden inside lowering or a backend. + HIR validation rejects duplicate and unresolved symbols, invalid declaration spans, and schemes inconsistent with their values. LIR validation rechecks artifact-local references and spans, then verifies closure calls, branches and decision guards, primitive signatures, aggregate representations, declaration value types, intrinsic catalog membership, and complete target-import linkage. Artifact deserialization runs the same LIR validator, so malformed JSON cannot bypass compiler invariants. Lexical `let` is retained as a structured LIR node. Its symbol is visible only in the body, or in both the binding and body for recursive functions. The interpreter closes the recursive environment explicitly; Go emission declares the function slot before assigning its closure. `raise` becomes an explicit `Fail` node so consumers preserve evaluation order and attach their own stage-specific failure diagnostic. diff --git a/src/FScript.Compiler/Compiler.fs b/src/FScript.Compiler/Compiler.fs index 525659d..a4d2b4b 100644 --- a/src/FScript.Compiler/Compiler.fs +++ b/src/FScript.Compiler/Compiler.fs @@ -40,7 +40,7 @@ module Compiler = |> HirLowering.lower externs |> Result.bind HirValidation.validate |> Result.bind (fun hir -> - hir |> LirLowering.lower |> LirValidation.validate + hir |> LirLowering.lower |> LirOptimization.optimize |> LirValidation.validate |> Result.map (fun lir -> { Hir = hir; Lir = lir })) with | TypeException error -> diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index 615d513..0902d6d 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -325,3 +325,8 @@ module LirValidation = program.Declarations |> List.iter (fun declaration -> checkSpan declaration.Span; checkExpression Set.empty declaration.Value; if not (compatible declaration.Type declaration.Value.Type) then report "FSL2004" $"LIR declaration '{declaration.Name}' type is inconsistent." declaration.Span) program.Entry |> Option.iter (checkExpression Set.empty) if diagnostics.Count = 0 then Ok program else Error(List.ofSeq diagnostics) + +module LirOptimization = + /// The initial pipeline deliberately performs no optimization. Keeping this + /// pass explicit makes future transformations independently testable. + let optimize (program: Lir.Program) = program diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index 36b78c0..804a422 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -47,3 +47,9 @@ type HirTests() = match LirValidation.validate { Declarations = []; Entry = Some intrinsic } with | Ok _ -> Assert.Fail("Expected unavailable primitive diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSL2009")) + + [] + member _.``Initial LIR optimization pass is identity`` () = + let expression = ({ Type = TInt; Span = span; Node = Lir.Constant(Lir.Int 1L) }: Lir.Expression) + let program: Lir.Program = { Declarations = []; Entry = Some expression } + Assert.That(LirOptimization.optimize program = program, Is.True) From 8c8bde8dd732f074ed55ebc68c67a709ab1a2186 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 18:37:36 +0200 Subject: [PATCH 035/133] feat(go): preserve tuple representations --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 ++ docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Backend.Go/GoBackend.fs | 16 +++++++++++----- src/FScript.Backend.Go/Runtime/closures.go | 2 +- src/FScript.Backend.Go/Runtime/patterns.go | 2 +- src/FScript.Backend.Go/Runtime/runtime.go | 8 ++++++++ tests/FScript.Backend.Go.Tests/GoBackendTests.fs | 1 + .../ConformanceTests.fs | 1 + 9 files changed, 27 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e428a1d..0198be7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added a distinct generated-Go tuple representation with tuple formatting, pattern matching, map-pair handling, and cross-runtime coverage. - Added an explicit identity LIR optimization pass between lowering and validation for future semantic transformations. - Embedded preformatted Go runtime templates and made every generated project pass a zero-diff `gofmt` gate without production toolchain dependence. - Added portable `Console.writeLine` execution and stdout capture across LIR and generated Go conformance runs. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index a3cecf4..c516bd4 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -66,6 +66,8 @@ Artifact metadata records the actual compiler and language assembly versions, no - Structural equality and record conversions are explicit LIR operations/helpers; Go `==` and Go nominal assignability never define FScript semantics. - LIR remains structured and non-SSA until a concrete optimization requirement justifies a control-flow IR. +Generated Go uses `fsruntime.Tuple` for tuple shapes and `[]any` only for erased lists. This preserves tuple/list formatting and prevents tuple patterns or `(key, value)` map inputs from depending on a shared slice representation. + ## Runtime and foreign calls Compiler bindings classify names as portable intrinsics, backend runtime primitives, target imports, or unsupported host externs. This metadata is distinct from executable .NET `ExternalFunction` implementations. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index f075310..1948a3e 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -47,6 +47,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Compiler extern input is declarative: it carries a name, type scheme, arity, and linkage classification, never an executable `.NET` delegate. Host execution values remain confined to the legacy/runtime API. - `Console.writeLine` evaluates its string argument once, appends one stdout line, and returns unit. LIR reports captured lines in order through `ExecutionResult.Stdout`; generated Go emits them in the same order before its final formatted result. - Generated Go source is deterministic and already canonical according to `gofmt`; generation itself has no installed-Go dependency. Local integration tests skip only when Go is unavailable, while CI installs Go 1.25 and treats its absence as a failure. +- Tuples and lists remain observably distinct in generated Go: tuples format with parentheses, lists with brackets, tuple patterns require tuple values, and map pair inputs consume tuple values. ## Builtins, externs, attributes, and tasks diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index 557ff18..c9bb438 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -35,7 +35,7 @@ module GoBackend = | TTypeToken -> "string" | TVar _ -> "any" | TFun _ -> "fsruntime.Closure" - | TTuple _ -> "[]any" + | TTuple _ -> "fsruntime.Tuple" | TList _ -> "[]any" | TOption _ -> "fsruntime.Option" | TRecord _ -> "map[string]any" @@ -75,7 +75,8 @@ module GoBackend = | TFloat -> $"fsruntime.Float({code})" | TBool -> $"fsruntime.Bool({code})" | TString -> $"fsruntime.String({code})" - | TList _ | TTuple _ -> $"fsruntime.List({code})" + | TList _ -> $"fsruntime.List({code})" + | TTuple _ -> $"fsruntime.AsTuple({code})" | TRecord _ -> $"fsruntime.Record({code})" | TOption _ -> $"fsruntime.AsOption({code})" | TNamed "Environment" -> $"fsruntime.Record({code})" @@ -157,8 +158,8 @@ module GoBackend = let test, assignments = combine [ headPart; tailPart ] $"len(fsruntime.List({valueCode})) > 0 && ({test})", assignments | Lir.TuplePattern items -> - items |> List.mapi (fun index item -> emitPattern $"fsruntime.List({valueCode})[{index}]" item) - |> combine |> fun (test, assignments) -> $"len(fsruntime.List({valueCode})) == {items.Length} && ({test})", assignments + items |> List.mapi (fun index item -> emitPattern $"fsruntime.AsTuple({valueCode})[{index}]" item) + |> combine |> fun (test, assignments) -> $"len(fsruntime.AsTuple({valueCode})) == {items.Length} && ({test})", assignments | Lir.RecordPattern fields -> fields |> List.map (fun (name, item) -> let quoted = System.Text.Json.JsonSerializer.Serialize(name) @@ -248,7 +249,12 @@ module GoBackend = | _ -> Ok $"({l} {token} {r})" | None -> Error $"Unsupported Go operation {op}" | Error error, _ | _, Error error -> Error error - | Lir.Tuple items | Lir.List items -> + | Lir.Tuple items -> + items |> List.map (emitExpression environment) |> collect + |> Result.map (fun values -> + let joined = String.concat ", " values + $"fsruntime.Tuple{{{joined}}}") + | Lir.List items -> items |> List.map (emitExpression environment) |> collect |> Result.map (fun values -> let joined = String.concat ", " values diff --git a/src/FScript.Backend.Go/Runtime/closures.go b/src/FScript.Backend.Go/Runtime/closures.go index b605373..0793b47 100644 --- a/src/FScript.Backend.Go/Runtime/closures.go +++ b/src/FScript.Backend.Go/Runtime/closures.go @@ -140,7 +140,7 @@ func DispatchIntrinsic(name string, args []any) any { case "Map.ofList": values := []MapEntry{} for _, pair := range List(args[0]) { - tuple := List(pair) + tuple := AsTuple(pair) values = mapSet(values, tuple[0], tuple[1], true) } return OrderedMap{Entries: values} diff --git a/src/FScript.Backend.Go/Runtime/patterns.go b/src/FScript.Backend.Go/Runtime/patterns.go index a4f5179..0532daf 100644 --- a/src/FScript.Backend.Go/Runtime/patterns.go +++ b/src/FScript.Backend.Go/Runtime/patterns.go @@ -73,7 +73,7 @@ func matchRuntimePattern(value any, pattern Pattern) (map[int]any, bool) { } return mergePatternBindings(left, right) case "tuple": - values, ok := value.([]any) + values, ok := value.(Tuple) if !ok || len(values) != len(pattern.Items) { return nil, false } diff --git a/src/FScript.Backend.Go/Runtime/runtime.go b/src/FScript.Backend.Go/Runtime/runtime.go index 7461454..ac4fb42 100644 --- a/src/FScript.Backend.Go/Runtime/runtime.go +++ b/src/FScript.Backend.Go/Runtime/runtime.go @@ -12,6 +12,7 @@ type Option struct { HasValue bool Value any } +type Tuple []any type Union struct { TypeName string CaseName string @@ -36,6 +37,7 @@ func Float(value any) float64 { return value.(float64) } func Bool(value any) bool { return value.(bool) } func String(value any) string { return value.(string) } func List(value any) []any { return value.([]any) } +func AsTuple(value any) Tuple { return value.(Tuple) } func Record(value any) map[string]any { return value.(map[string]any) } func AsOption(value any) Option { return value.(Option) } func AsUnion(value any) Union { return value.(Union) } @@ -271,6 +273,12 @@ func Format(value any) string { parts[index] = Format(value) } return "[" + strings.Join(parts, ";") + "]" + case Tuple: + parts := make([]string, len(item)) + for index, value := range item { + parts[index] = Format(value) + } + return "(" + strings.Join(parts, ", ") + ")" default: return fmt.Sprint(value) } diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index ff49a0c..2bd3456 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -35,6 +35,7 @@ type GoBackendTests() = [] [] [] + [] [] [] [ x + 1\n | None -> 0", "4")>] diff --git a/tests/FScript.Conformance.Tests/ConformanceTests.fs b/tests/FScript.Conformance.Tests/ConformanceTests.fs index 80c686c..fc1726a 100644 --- a/tests/FScript.Conformance.Tests/ConformanceTests.fs +++ b/tests/FScript.Conformance.Tests/ConformanceTests.fs @@ -98,6 +98,7 @@ type ConformanceTests() = [] [ 0 | Circle radius -> radius")>] [] + [] [ state + value) 0 [1; 2; 3]")>] [] [ value | None -> 0")>] From 0c7b8fe4c7484c97675ee42e1f2ce8ed2dcc3e81 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 18:45:50 +0200 Subject: [PATCH 036/133] feat(go): adapt imported record bindings --- CHANGELOG.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 + docs/specs/compiler-semantics-baseline.md | 1 + docs/specs/syntax-and-indentation.md | 7 +++ src/FScript.Backend.Go/GoBackend.fs | 54 ++++++++++++++++--- src/FScript.Compiler/Hir.fs | 11 +++- src/FScript.Compiler/HirLowering.fs | 10 +++- src/FScript.Compiler/HirValidation.fs | 7 ++- src/FScript.Compiler/IrArtifact.fs | 31 +++++++---- src/FScript.Compiler/Lir.fs | 25 +++++++-- src/FScript.Language/Ast.fs | 11 ++-- src/FScript.Language/BuiltinTypes.fs | 2 + src/FScript.Language/Parser.fs | 13 +++-- .../GoBackendTests.fs | 28 ++++++++++ tests/FScript.Compiler.Tests/HirTests.fs | 8 +-- tests/FScript.Language.Tests/ParserTests.fs | 9 ++++ 16 files changed, 187 insertions(+), 33 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0198be7..ee65c14 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added imported record declarations with serialized HIR/LIR metadata, primitive Go boundary adapters, and explicit unsupported-shape diagnostics. - Added a distinct generated-Go tuple representation with tuple formatting, pattern matching, map-pair handling, and cross-runtime coverage. - Added an explicit identity LIR optimization pass between lowering and validation for future semantic transformations. - Embedded preformatted Go runtime templates and made every generated project pass a zero-diff `gofmt` gate without production toolchain dependence. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index c516bd4..bcdbfa5 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -82,6 +82,8 @@ The LIR interpreter rejects reachable target imports it cannot supply. The Go ba Initial Go imports use `[] let name : complete -> signature` with no body. The frontend requires a function type and records linkage separately from its FScript signature. Go emission uses deterministic package aliases and currently accepts only primitive/unit arguments and results; other representations receive `FSB4003`. The LIR interpreter keeps unused imports inert and reports `FSI3023` only when one is fully applied. +Imported Go record types use `[] type Name = { Field: primitive; ... }`. Their module/type linkage and ordered field schema survive HIR, LIR, and `.fsir` serialization. At imported function boundaries, generated Go converts FScript record maps to the declared Go struct and converts returned structs back to record maps. Fields must begin with an uppercase letter and currently use only unit, integer, floating-point, Boolean, or string representations; imported unions and other field shapes receive `FSB4005`. + ## Cross-runtime conformance The conformance suite evaluates one source program through the legacy evaluator, a serialized-and-deserialized LIR program, and a generated Go executable. It compares successful results and failure categories across nested structural equality, named/structural record conversion, unions, higher-order and polymorphic functions, mutual recursion, and ordered maps. Named semantic types are resolved to their record or union representation while HIR is built, so a named record crossing a function boundary cannot accidentally receive the union representation in a downstream consumer. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 1948a3e..415e8d4 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -48,6 +48,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - `Console.writeLine` evaluates its string argument once, appends one stdout line, and returns unit. LIR reports captured lines in order through `ExecutionResult.Stdout`; generated Go emits them in the same order before its final formatted result. - Generated Go source is deterministic and already canonical according to `gofmt`; generation itself has no installed-Go dependency. Local integration tests skip only when Go is unavailable, while CI installs Go 1.25 and treats its absence as a failure. - Tuples and lists remain observably distinct in generated Go: tuples format with parentheses, lists with brackets, tuple patterns require tuple values, and map pair inputs consume tuple values. +- Backend-imported records retain exact field names and declaration linkage. The initial Go adapter accepts exported primitive fields only, performs record/struct conversion solely at target-call boundaries, and rejects imported unions, empty schemas, lowercase fields, tasks, and aggregate/function fields with `FSB4005`. ## Builtins, externs, attributes, and tasks diff --git a/docs/specs/syntax-and-indentation.md b/docs/specs/syntax-and-indentation.md index fcdca71..700514f 100644 --- a/docs/specs/syntax-and-indentation.md +++ b/docs/specs/syntax-and-indentation.md @@ -245,3 +245,10 @@ let ops = - imports are effectively deduplicated. - `import` must appear before executable/type code in a file. - Parse/type/eval errors include file-aware spans (`file`, `line`, `column`) so failures in imported files point to the offending source. + +## Backend import attributes + +- A bodyless target function import uses `[] let name : complete -> signature`. +- An imported record schema uses `[] type Name = { Field: type; ... }`. +- `Import` takes exactly two non-empty arguments and is valid only on a top-level type declaration or bodyless typed `let`. +- Imported record field spelling is preserved exactly for backend adapters; the initial Go backend requires exported, uppercase primitive fields. diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index c9bb438..2d64682 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -3,12 +3,16 @@ namespace FScript.Compiler open System open System.IO open System.Text +open System.Threading open FScript.Language type GoOptions = { ModuleName: string } type GeneratedFile = { RelativePath: string; Content: string } module GoBackend = + let private currentImportedRecords = AsyncLocal option>() + let private tryImportedRecord name = + currentImportedRecords.Value |> Option.bind (Map.tryFind name) let private indent (value: string) = value.Replace("\n", "\n\t") let private runtimeTemplate fileName = @@ -40,6 +44,7 @@ module GoBackend = | TOption _ -> "fsruntime.Option" | TRecord _ -> "map[string]any" | TNamed "Environment" -> "map[string]any" + | TNamed name when tryImportedRecord name |> Option.isSome -> "map[string]any" | TUnion _ | TNamed _ -> "fsruntime.Union" | TMap _ -> "fsruntime.OrderedMap" | value -> invalidArg "value" $"Unsupported Go representation for {Types.typeToString value}." @@ -80,6 +85,7 @@ module GoBackend = | TRecord _ -> $"fsruntime.Record({code})" | TOption _ -> $"fsruntime.AsOption({code})" | TNamed "Environment" -> $"fsruntime.Record({code})" + | TNamed name when tryImportedRecord name |> Option.isSome -> $"fsruntime.Record({code})" | TUnion _ | TNamed _ -> $"fsruntime.AsUnion({code})" | TMap _ -> $"fsruntime.AsMap({code})" | TFun _ -> $"fsruntime.AsClosure({code})" @@ -198,17 +204,38 @@ module GoBackend = let private emitTargetImport modulePath memberName importType = let alias = importAlias modulePath + let importedArgument raw _ (record: Lir.ImportedRecord) = + let fields = + record.Fields + |> List.map (fun (fieldName, fieldType) -> + let fieldAccess = $"fsruntime.Record({raw})[{System.Text.Json.JsonSerializer.Serialize(fieldName)}]" + let fieldValue = cast fieldType fieldAccess + $"{fieldName}: {fieldValue}") + |> String.concat ", " + $"{importAlias record.ModulePath}.{record.MemberName}{{{fields}}}" + let importedResult code (record: Lir.ImportedRecord) = + let fields = + record.Fields + |> List.map (fun (fieldName, _) -> $"{System.Text.Json.JsonSerializer.Serialize(fieldName)}: importedResult.{fieldName}") + |> String.concat ", " + $"func() map[string]any {{\n\timportedResult := {code}\n\treturn map[string]any{{{fields}}}\n}}()" let rec build index arguments = function | TFun(argumentType, resultType) -> let raw = $"targetArg{index}" let argument = $"targetValue{index}" let body = build (index + 1) (arguments @ [ argument ]) resultType - $"fsruntime.Closure{{Call: func({raw} any) any {{\n\t{argument} := {cast argumentType raw}\n\treturn {indent body}\n}}}}" + let argumentValue = + match argumentType with + | TNamed name -> tryImportedRecord name |> Option.map (importedArgument raw name) |> Option.defaultValue (cast argumentType raw) + | _ -> cast argumentType raw + $"fsruntime.Closure{{Call: func({raw} any) any {{\n\t{argument} := {argumentValue}\n\treturn {indent body}\n}}}}" | resultType -> let joined = String.concat ", " arguments + let call = $"{alias}.{memberName}({joined})" match resultType with | TUnit -> $"func() any {{ {alias}.{memberName}({joined}); return struct{{}}{{}} }}()" - | _ -> $"{alias}.{memberName}({joined})" + | TNamed name -> tryImportedRecord name |> Option.map (importedResult call) |> Option.defaultValue call + | _ -> call build 0 [] importType let rec private emitExpression environment (expression: Lir.Expression) = @@ -466,6 +493,7 @@ func Environment() map[string]any { let private generateCore options (program: Lir.Program) : Result = try + currentImportedRecords.Value <- Some(program.ImportedRecords |> List.map (fun record -> record.Name, record) |> Map.ofList) let targetImports = program.Declarations |> List.choose (fun declaration -> match declaration.Value.Node with Lir.TargetImport(modulePath, memberName) -> Some(modulePath, memberName, declaration.Type, declaration.Span) | _ -> None) @@ -473,6 +501,9 @@ func Environment() map[string]any { let rec directlyRepresentable = function | TUnit | TInt | TFloat | TBool | TString -> true | TFun(argument, result) -> directlyRepresentable argument && directlyRepresentable result + | TNamed name -> + tryImportedRecord name + |> Option.exists (fun record -> not record.IsUnion && record.Fields |> List.forall (snd >> directlyRepresentable)) | _ -> false let unsupportedImport = targetImports |> List.tryFind (fun (_, _, importType, _) -> not (directlyRepresentable importType)) let names = @@ -503,7 +534,8 @@ func Environment() map[string]any { let moduleName = if String.IsNullOrWhiteSpace options.ModuleName then "fscript.generated/program" else options.ModuleName let importLines = ($"\t\"{moduleName}/internal/fsruntime\"") :: - (targetImports |> List.map (fun (modulePath, _, _, _) -> modulePath) |> List.distinct |> List.map (fun path -> $"\t{importAlias path} {System.Text.Json.JsonSerializer.Serialize(path)}")) + ([ yield! targetImports |> List.map (fun (modulePath, _, _, _) -> modulePath) + yield! program.ImportedRecords |> List.map _.ModulePath ] |> List.distinct |> List.sort |> List.map (fun path -> $"\t{importAlias path} {System.Text.Json.JsonSerializer.Serialize(path)}")) |> String.concat "\n" let programSource = $"package fscriptprogram\n\nimport (\n{importLines}\n)\n\n{declarations}\nfunc init() {{\n{initializers}}}\n\nfunc Run() any {{ return {entryCode} }}\n" let printExpression = @@ -522,6 +554,7 @@ func Environment() map[string]any { Error [ { Code = "FSB4002"; Stage = GoGeneration; Message = ex.Message; Span = None } ] let generate options (program: Lir.Program) : Result = + currentImportedRecords.Value <- Some(program.ImportedRecords |> List.map (fun record -> record.Name, record) |> Map.ofList) let rec containsTask = function | TTask _ -> true | TFun(argument, result) | TMap(argument, result) -> containsTask argument || containsTask result @@ -561,6 +594,15 @@ func Environment() map[string]any { program.Declarations |> List.tryPick (fun declaration -> if Set.contains declaration.Id reachable && containsTask declaration.Type then Some declaration.Span else None) |> Option.orElseWith (fun () -> program.Entry |> Option.bind (fun entry -> if containsTask entry.Type then Some entry.Span else None)) - match taskSpan with - | Some span -> Error [ { Code = "FSB4004"; Stage = GoGeneration; Message = "Task values are not supported by the Go backend."; Span = Some span } ] - | None -> generateCore options program + let invalidImportedRecord = + program.ImportedRecords + |> List.tryFind (fun record -> + record.IsUnion + || record.Fields.IsEmpty + || (record.Fields |> List.exists (fun (name, fieldType) -> + let unsupportedType = match fieldType with TUnit | TInt | TFloat | TBool | TString -> false | _ -> true + System.String.IsNullOrEmpty(name) || not (System.Char.IsUpper(name.[0])) || containsTask fieldType || unsupportedType))) + match invalidImportedRecord, taskSpan with + | Some record, _ -> Error [ { Code = "FSB4005"; Stage = GoGeneration; Message = $"Imported Go record '{record.Name}' must have exported, exactly named primitive fields and cannot be a union."; Span = Some record.Span } ] + | None, Some span -> Error [ { Code = "FSB4004"; Stage = GoGeneration; Message = "Task values are not supported by the Go backend."; Span = Some span } ] + | None, None -> generateCore options program diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs index 85dc277..1a5ae8b 100644 --- a/src/FScript.Compiler/Hir.fs +++ b/src/FScript.Compiler/Hir.fs @@ -109,8 +109,17 @@ module Hir = Span: Span Value: Expression } + type ImportedRecord = + { Name: string + ModulePath: string + MemberName: string + IsUnion: bool + Fields: (string * Type) list + Span: Span } + type Program = - { Declarations: Declaration list + { ImportedRecords: ImportedRecord list + Declarations: Declaration list Entry: Expression option } let symbolNumber (SymbolId value) = value diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index 890a3bc..5f26262 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -481,5 +481,13 @@ module HirLowering = Span = span Value = { Type = importedType; Span = span; Node = Hir.TargetImport(import.ModulePath, import.MemberName) } }: Hir.Declaration) ] | TypeInfer.TSType _ -> () - if errors.IsEmpty then Ok { Declarations = declarations; Entry = entry } + let importedRecords = + typeDefinitions + |> List.choose (fun definition -> + definition.Import + |> Option.map (fun import -> + ({ Name = definition.Name; ModulePath = import.ModulePath; MemberName = import.MemberName; IsUnion = not definition.Cases.IsEmpty + Fields = definition.Fields |> List.map (fun (name, fieldType) -> name, typeFromRef fieldType) + Span = definition.Span }: Hir.ImportedRecord))) + if errors.IsEmpty then Ok { ImportedRecords = importedRecords; Declarations = declarations; Entry = entry } else Error errors diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index 36c5980..056453f 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -23,7 +23,12 @@ module HirValidation = let validate (program: Hir.Program) = let declarationIds = program.Declarations |> List.map (fun value -> value.Id) let duplicateIds = declarationIds |> List.countBy id |> List.filter (fun (_, count) -> count > 1) - if not duplicateIds.IsEmpty then + let invalidImports = + program.ImportedRecords |> List.exists (fun record -> System.String.IsNullOrWhiteSpace(record.ModulePath) || System.String.IsNullOrWhiteSpace(record.MemberName)) + || (program.ImportedRecords |> List.map _.Name |> List.distinct |> List.length) <> program.ImportedRecords.Length + if invalidImports then + Error [ Diagnostic.hir "FSH1105" "HIR imported record linkage must be complete and uniquely named." None ] + elif not duplicateIds.IsEmpty then Error [ Diagnostic.hir "FSH1101" "HIR declaration IDs must be unique." None ] elif program.Declarations |> List.exists (fun declaration -> not (validSpan declaration.Span && validSpan declaration.Value.Span)) then Error [ Diagnostic.hir "FSH1103" "HIR contains an invalid source span." None ] diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index aef3d26..c8ac4ca 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -19,7 +19,8 @@ module IrArtifact = let private sourceMetadata (program: Lir.Program) = let paths = - [ yield! program.Declarations |> List.choose (fun declaration -> declaration.Span.Start.File) + [ yield! program.ImportedRecords |> List.choose (fun declaration -> declaration.Span.Start.File) + yield! program.Declarations |> List.choose (fun declaration -> declaration.Span.Start.File) match program.Entry with Some entry -> yield! entry.Span.Start.File |> Option.toList | None -> () ] |> List.distinct |> List.sort @@ -34,12 +35,12 @@ module IrArtifact = else None) let private dependencies (program: Lir.Program) = - program.Declarations - |> List.choose (fun declaration -> - match declaration.Value.Node with - | Lir.HostExternal(name, _) -> Some $"host:{name}" - | Lir.TargetImport(modulePath, memberName) -> Some $"target:{modulePath}:{memberName}" - | _ -> None) + [ yield! program.ImportedRecords |> List.map (fun record -> $"target:{record.ModulePath}:{record.MemberName}") + yield! program.Declarations |> List.choose (fun declaration -> + match declaration.Value.Node with + | Lir.HostExternal(name, _) -> Some $"host:{name}" + | Lir.TargetImport(modulePath, memberName) -> Some $"target:{modulePath}:{memberName}" + | _ -> None) ] |> List.distinct |> List.sort @@ -253,7 +254,14 @@ module IrArtifact = writer.WriteEndArray(); writer.WriteStartArray("dependencies") dependencies program |> List.iter writer.WriteStringValue writer.WriteEndArray() - writer.WriteStartObject("program"); writer.WriteStartArray("declarations") + writer.WriteStartObject("program"); writer.WriteStartArray("importedRecords") + for record in program.ImportedRecords do + writer.WriteStartObject(); writer.WriteString("name", record.Name); writer.WriteString("modulePath", record.ModulePath); writer.WriteString("memberName", record.MemberName); writer.WriteBoolean("union", record.IsUnion) + writer.WriteStartArray("fields") + for name, fieldType in record.Fields do + writer.WriteStartObject(); writer.WriteString("name", name); writer.WritePropertyName("type"); writeType writer fieldType; writer.WriteEndObject() + writer.WriteEndArray(); writer.WritePropertyName("span"); writeSpan writer record.Span; writer.WriteEndObject() + writer.WriteEndArray(); writer.WriteStartArray("declarations") for declaration in program.Declarations do writer.WriteStartObject(); writer.WriteNumber("id", Lir.symbolNumber declaration.Id); writer.WriteString("name", declaration.Name) writer.WritePropertyName("type"); writeType writer declaration.Type; writer.WriteBoolean("recursive", declaration.IsRecursive) @@ -421,13 +429,18 @@ module IrArtifact = Error [ { Code = "FSA5005"; Stage = Artifact; Message = "IR artifact metadata is invalid or non-canonical."; Span = None } ] else let programElement = property "program" root + let importedRecords = (property "importedRecords" programElement).EnumerateArray() |> Seq.map (fun item -> + ({ Name = stringProperty "name" item; ModulePath = stringProperty "modulePath" item; MemberName = stringProperty "memberName" item; IsUnion = (property "union" item).GetBoolean() + Fields = (property "fields" item).EnumerateArray() |> Seq.map (fun field -> stringProperty "name" field, readType (property "type" field)) |> Seq.toList + Span = readSpan (property "span" item) }: Lir.ImportedRecord)) |> Seq.toList let declarations = (property "declarations" programElement).EnumerateArray() |> Seq.map (fun (item: JsonElement) -> ({ Id = Lir.SymbolId((property "id" item).GetInt32()); Name = stringProperty "name" item; Type = readType (property "type" item) IsRecursive = (property "recursive" item).GetBoolean(); IsExported = (property "exported" item).GetBoolean() Span = readSpan (property "span" item); Value = readExpression (property "value" item) }: Lir.Declaration)) |> Seq.toList let entryElement = property "entry" programElement let program : Lir.Program = - { Declarations = declarations + { ImportedRecords = importedRecords + Declarations = declarations Entry = if entryElement.ValueKind = JsonValueKind.Null then None else Some(readExpression entryElement) } LirValidation.validate program with ex -> Error [ { Code = "FSA5003"; Stage = Artifact; Message = ex.Message; Span = None } ] diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index 0902d6d..6671d55 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -97,8 +97,17 @@ module Lir = Span: Span Value: Expression } + type ImportedRecord = + { Name: string + ModulePath: string + MemberName: string + IsUnion: bool + Fields: (string * Type) list + Span: Span } + type Program = - { Declarations: Declaration list + { ImportedRecords: ImportedRecord list + Declarations: Declaration list Entry: Expression option } let symbolNumber (SymbolId value) = value @@ -180,7 +189,12 @@ module LirLowering = { Type = value.Type; Span = value.Span; Node = node } let lower (program: Hir.Program) : Lir.Program = - { Declarations = + { ImportedRecords = + program.ImportedRecords + |> List.map (fun value -> + ({ Name = value.Name; ModulePath = value.ModulePath; MemberName = value.MemberName; IsUnion = value.IsUnion + Fields = value.Fields; Span = value.Span }: Lir.ImportedRecord)) + Declarations = program.Declarations |> List.map (fun value -> ({ Id = symbol value.Id; Name = value.Name; Type = value.Value.Type @@ -208,7 +222,12 @@ module LirValidation = let validate (program: Lir.Program) = let ids = program.Declarations |> List.map _.Id - if ids.Length <> (ids |> Set.ofList |> Set.count) then + let invalidImports = + program.ImportedRecords |> List.exists (fun record -> System.String.IsNullOrWhiteSpace(record.ModulePath) || System.String.IsNullOrWhiteSpace(record.MemberName)) + || (program.ImportedRecords |> List.map _.Name |> List.distinct |> List.length) <> program.ImportedRecords.Length + if invalidImports then + Error [ { Code = "FSL2012"; Stage = Lowering; Message = "LIR imported record linkage must be complete and uniquely named."; Span = None } ] + elif ids.Length <> (ids |> Set.ofList |> Set.count) then Error [ { Code = "FSL2001"; Stage = Lowering; Message = "LIR declaration IDs must be unique."; Span = None } ] else let diagnostics = ResizeArray() diff --git a/src/FScript.Language/Ast.fs b/src/FScript.Language/Ast.fs index 653f016..0764290 100644 --- a/src/FScript.Language/Ast.fs +++ b/src/FScript.Language/Ast.fs @@ -1,10 +1,16 @@ namespace FScript.Language +type BackendImport = + { ModulePath: string + MemberName: string + Span: Span } + type TypeDef = { Name: string IsRecursive: bool Fields: (string * TypeRef) list Cases: (string * TypeRef option) list + Import: BackendImport option Span: Span } and TypeRef = @@ -25,11 +31,6 @@ type Attribute = Arguments: string list Span: Span } -type BackendImport = - { ModulePath: string - MemberName: string - Span: Span } - type Literal = | LInt of int64 | LFloat of float diff --git a/src/FScript.Language/BuiltinTypes.fs b/src/FScript.Language/BuiltinTypes.fs index f807db5..bef862c 100644 --- a/src/FScript.Language/BuiltinTypes.fs +++ b/src/FScript.Language/BuiltinTypes.fs @@ -10,6 +10,7 @@ module BuiltinTypes = [ "ScriptName", TRPostfix(TRName "string", "option") "Arguments", TRPostfix(TRName "string", "list") ] Cases = [] + Import = None Span = unknownSpan } let fsKindType : TypeDef = @@ -20,6 +21,7 @@ module BuiltinTypes = [ "File", Some (TRName "string") "Directory", Some (TRName "string") "Missing", None ] + Import = None Span = unknownSpan } let typeDefs : TypeDef list = diff --git a/src/FScript.Language/Parser.fs b/src/FScript.Language/Parser.fs index 394b35f..269794a 100644 --- a/src/FScript.Language/Parser.fs +++ b/src/FScript.Language/Parser.fs @@ -346,7 +346,7 @@ module Parser = argsDone <- true args - and parseTypeDecl () : Stmt = + and parseTypeDecl (backendImport: Attribute option) : Stmt = let typeTok = stream.Expect(Type, "Expected 'type'") let isRec = stream.Match(Rec) let nameTok = stream.ExpectIdent("Expected identifier after 'type'") @@ -458,7 +458,14 @@ module Parser = if not (stream.Match(Dedent)) then raise (ParseException { Message = "Expected dedent after type declaration"; Span = stream.Peek().Span }) while stream.Match(Dedent) do () - SType { Name = name; IsRecursive = isRec; Fields = fields; Cases = cases; Span = mkSpanFrom typeTok.Span endSpan } + let import = + backendImport + |> Option.map (fun attribute -> + match attribute.Arguments with + | [ modulePath; memberName ] when not (System.String.IsNullOrWhiteSpace modulePath) && not (System.String.IsNullOrWhiteSpace memberName) -> + { ModulePath = modulePath; MemberName = memberName; Span = attribute.Span } + | _ -> raise (ParseException { Message = "Import requires module path and member name: []"; Span = attribute.Span })) + SType { Name = name; IsRecursive = isRec; Fields = fields; Cases = cases; Import = import; Span = mkSpanFrom typeTok.Span endSpan } and parsePattern () : Pattern = stream.SkipNewlines() @@ -1661,7 +1668,7 @@ module Parser = | Type -> if isExported then raise (ParseException { Message = "'[]' is only valid for top-level let bindings"; Span = stream.Peek().Span }) - parseTypeDecl() + parseTypeDecl backendImport | Let -> match backendImport with | Some attribute when isExported -> raise (ParseException { Message = "Import and Export cannot be combined initially"; Span = attribute.Span }) diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index 2bd3456..91e8b74 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -32,6 +32,34 @@ type GoBackendTests() = | Ok _ -> Assert.Fail("Expected unsupported task diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSB4004")) + [] + member _.``Go backend emits imported record boundary adapters`` () = + let source = "[] type Person = { Name: string; Age: int }\n[] let birthday : Person -> Person\nbirthday { Name = \"Ada\"; Age = 42 }" + let compiled = + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Ok value -> value + | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + let roundTripped = + match compiled.Lir |> IrArtifact.serialize |> IrArtifact.deserialize with + | Ok value -> value + | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + Assert.That(roundTripped.ImportedRecords.Length, Is.EqualTo(1)) + match GoBackend.generate { ModuleName = "fscript.generated/importrecords" } roundTripped with + | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) + | Ok files -> + let program = files |> List.find (fun file -> file.RelativePath.EndsWith("program.go")) |> _.Content + Assert.That(program, Does.Contain("goimport")) + Assert.That(program, Does.Contain(".Person{Name:")) + Assert.That(program, Does.Contain("importedResult.Name")) + + [] + member _.``Go backend rejects unsupported imported record shapes`` () = + let source = "[] type Person = { name: string }\n1" + let compiled = match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with Ok value -> value | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + match GoBackend.generate { ModuleName = "fscript.generated/importrecords" } compiled.Lir with + | Ok _ -> Assert.Fail("Expected imported-record diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSB4005")) + [] [] [] diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index 804a422..b87f794 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -27,7 +27,7 @@ type HirTests() = let declaration: Hir.Declaration = { Id = Hir.SymbolId 0; Name = "value"; Scheme = Forall([], TString) IsRecursive = false; IsExported = false; Span = span; Value = value } - match HirValidation.validate { Declarations = [ declaration ]; Entry = None } with + match HirValidation.validate { ImportedRecords = []; Declarations = [ declaration ]; Entry = None } with | Ok _ -> Assert.Fail("Expected invalid HIR diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSH1104")) @@ -37,19 +37,19 @@ type HirTests() = let branch: Lir.Expression = { Type = TInt; Span = span Node = Lir.Branch(constant TInt (Lir.Int 1L), constant TInt (Lir.Int 2L), constant TString (Lir.String "no")) } - match LirValidation.validate { Declarations = []; Entry = Some branch } with + match LirValidation.validate { ImportedRecords = []; Declarations = []; Entry = Some branch } with | Ok _ -> Assert.Fail("Expected invalid LIR diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSL2007")) [] member _.``LIR validation rejects unknown primitives`` () = let intrinsic = ({ Type = TInt; Span = span; Node = Lir.Intrinsic "Unknown.value" }: Lir.Expression) - match LirValidation.validate { Declarations = []; Entry = Some intrinsic } with + match LirValidation.validate { ImportedRecords = []; Declarations = []; Entry = Some intrinsic } with | Ok _ -> Assert.Fail("Expected unavailable primitive diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSL2009")) [] member _.``Initial LIR optimization pass is identity`` () = let expression = ({ Type = TInt; Span = span; Node = Lir.Constant(Lir.Int 1L) }: Lir.Expression) - let program: Lir.Program = { Declarations = []; Entry = Some expression } + let program: Lir.Program = { ImportedRecords = []; Declarations = []; Entry = Some expression } Assert.That(LirOptimization.optimize program = program, Is.True) diff --git a/tests/FScript.Language.Tests/ParserTests.fs b/tests/FScript.Language.Tests/ParserTests.fs index cfe0884..cf85107 100644 --- a/tests/FScript.Language.Tests/ParserTests.fs +++ b/tests/FScript.Language.Tests/ParserTests.fs @@ -818,6 +818,15 @@ type ParserTests () = import.MemberName |> should equal "ToUpper" | _ -> Assert.Fail("Expected backend import") + [] + member _.``Parses backend import attributes on record types`` () = + match Helpers.parse "[] type Person = { Name: string; Age: int }" with + | [ SType { Name = "Person"; Import = Some import; Fields = fields } ] -> + import.ModulePath |> should equal "example.com/model" + import.MemberName |> should equal "Person" + fields.Length |> should equal 2 + | _ -> Assert.Fail("Expected imported record type") + [] member _.``Rejects backend imports with bodies`` () = let act () = Helpers.parse "[] let upper : string -> string = fun value -> value" |> ignore From 471000924fb2dffbec1e21b2c84341a1769360b4 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 18:55:39 +0200 Subject: [PATCH 037/133] refactor(language): expose recursive resolved expressions --- CHANGELOG.md | 2 + docs/architecture/typed-ir-and-backends.md | 4 +- docs/specs/compiler-semantics-baseline.md | 4 +- src/FScript.Backend.Go/GoBackend.fs | 2 + src/FScript.Compiler/Hir.fs | 2 + src/FScript.Compiler/HirLowering.fs | 25 ++++- src/FScript.Compiler/HirValidation.fs | 1 + src/FScript.Compiler/IrArtifact.fs | 4 + src/FScript.Compiler/Lir.fs | 3 + src/FScript.Interpreter.Lir/LirInterpreter.fs | 5 + src/FScript.Language/TypeInfer.fs | 92 ++++++++++++++++++- .../FScript.Compiler.Tests/IrArtifactTests.fs | 10 ++ .../ResolvedInferenceTests.fs | 23 +++++ 13 files changed, 168 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ee65c14..cc41431 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Exposed recursively typed resolved expressions and preserved explicit structural record conversions through HIR, LIR, and IR artifacts. + - Added imported record declarations with serialized HIR/LIR metadata, primitive Go boundary adapters, and explicit unsupported-shape diagnostics. - Added a distinct generated-Go tuple representation with tuple formatting, pattern matching, map-pair handling, and cross-runtime coverage. - Added an explicit identity LIR optimization pass between lowering and validation for future semantic transformations. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index bcdbfa5..260ebf8 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -10,7 +10,7 @@ source -> parser/includes -> inference + resolved typed tree -> HIR -> LIR +-> Go source backend ``` -`FScript.Language` remains the parser, type-system, legacy evaluator, and Fable-compatible package. It emits a resolved typed tree in the same inference pass as its compatibility `TypedProgram`. +`FScript.Language` remains the parser, type-system, legacy evaluator, and Fable-compatible package. It emits a recursive resolved typed tree in the same inference pass as its compatibility `TypedProgram`. Every resolved expression carries its final instantiated type and span plus optional lexical-reference, primitive-operation, and structural/named-record conversion evidence; compatibility inference and evaluation APIs continue to project their existing results. Resolved inference assigns deterministic program-local IDs to builtins, externs, types, union cases, declarations, parameters, pattern bindings, and locals in normalized source traversal. Declaration schemes and instantiated reference types normalize type variables from zero, so results do not depend on the process-global fresh-variable counter. The legacy `TypedProgram` remains a projection with unchanged public entry points. @@ -20,7 +20,7 @@ Validated LIR consumers are separate leaf assemblies. `FScript.Interpreter.Lir` ## IR boundaries -HIR is language-oriented and contains resolved artifact-local symbol identities, complete semantic types and schemes, source locations, declaration attributes, and explicit evidence for named/structural record conversions. It contains no Go types. +HIR is language-oriented and contains resolved artifact-local symbol identities, complete semantic types and schemes, source locations, declaration attributes, and explicit nodes for named/structural record conversions. Conversion evidence is produced by inference, not rediscovered from backend representations. HIR contains no Go types. LIR is executable but not a Go AST. It uses lifted functions, explicit captures and closure creation, resolved primitive operations, aggregate construction/projection, and structured control flow. Pattern matching becomes deterministic tests and projections before reaching a backend. Semantic types may remain alongside selected runtime representations for validation and diagnostics. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 415e8d4..c046545 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -9,7 +9,7 @@ This document records the behavior that the typed IR, LIR interpreter, and compi - Includes are resolved into one flattened program before inference. Alias-qualified declarations are rewritten by the include resolver; unresolved `SImport` nodes are rejected by inference. - Hindley-Milner inference generalizes top-level and local let bindings after applying substitutions. Recursive bindings are inferred monomorphically inside their recursive group and generalized afterward. - The active typed program stores top-level types but retains parser expressions in bodies. Compiler lowering therefore requires a new resolved, recursively typed result emitted by the inference pass; types cannot safely be reconstructed afterward. -- Resolved inference exposes deterministic symbols for every lexical binding category and links each captured reference to its target ID with its instantiated type. Type-variable numbering in resolved schemes and reference types is normalized independently of prior inference calls. +- Resolved inference exposes a recursive expression tree whose nodes carry their final instantiated type, span, lexical reference, resolved primitive operation, and named/structural record-coercion evidence. It also exposes deterministic symbols for every lexical binding category and links each captured reference to its target ID. Type-variable numbering in resolved schemes and reference types is normalized independently of prior inference calls. - Top-level statements execute in source order. Every non-final expression must have type `unit`; the final expression is the program result. - Compiled top-level expressions execute exactly once in source order even when interleaved with declarations. Top-level pattern bindings evaluate their scrutinee once, fail before later statements when unmatched, and expose all bound names to subsequent statements. @@ -28,7 +28,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Primitive runtime values are unit, signed 64-bit integer, IEEE-754 double, Boolean, and string. - Tuples and lists preserve element order. Options distinguish `None` from `Some value`. - Maps currently accept string keys through inference and have deterministic ordered-map behavior. Runtime support for integer map keys is legacy representation detail, not permission for newly compiled programs. -- Declared records retain declaration identity for named annotations. Structural record annotations retain field-shape compatibility and may accept additional fields where current inference permits it. A backend must insert representation conversions rather than change compatibility rules. +- Declared records retain declaration identity for named annotations. Structural record annotations retain field-shape compatibility and may accept additional fields where current inference permits it. Inference records coercion evidence at the accepting application; HIR and LIR retain an explicit conversion node so backends convert representations without changing compatibility rules. - Discriminated unions retain their declared type name, case name, and optional payload. Recursive types require `type rec`; mutually recursive type-definition graphs are rejected. - `typeof` produces an FScript type token and `nameof` produces the resolved source name. - Compiled local pattern bindings evaluate their source once and use the ordinary match-failure path. Compiled `for` evaluates its list once, executes bodies from first to last, discards their values, and returns unit. Compiled `typeof` retains the resolved declared record or union shape in the artifact instead of performing a name lookup at execution time. diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index 2d64682..109d074 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -356,6 +356,7 @@ module GoBackend = let joined = String.concat ", " values Ok $"fsruntime.UpdateRecord({targetCode}, map[string]any{{{joined}}})" | Error error, _ | _, Error error -> Error error + | Lir.RecordConvert(target, _) -> emitExpression environment target | Lir.IndexGet(target, index) -> match emitExpression environment target, emitExpression environment index with | Ok targetCode, Ok indexCode -> Ok $"fsruntime.Index({targetCode}, {indexCode})" @@ -576,6 +577,7 @@ func Environment() map[string]any { | Lir.LetRecursiveGroup(bindings, body) -> references body :: (bindings |> List.map (fun (_, _, _, value) -> references value)) |> Set.unionMany | Lir.DecisionTree(target, cases) -> references target :: (cases |> List.collect (fun case -> [ yield! case.Guard |> Option.map references |> Option.toList; references case.Body ])) |> Set.unionMany | Lir.RecordUpdate(target, fields) -> references target :: (fields |> List.map (snd >> references)) |> Set.unionMany + | Lir.RecordConvert(target, _) -> references target | Lir.Interpolate parts -> parts |> List.choose (function Lir.TextPart _ -> None | Lir.ValuePart value -> Some(references value)) |> Set.unionMany | Lir.UnionCase(_, _, payload) -> payload |> Option.map references |> Option.defaultValue Set.empty | Lir.Map entries -> entries |> List.map (function Lir.KeyValue(key, value) -> Set.union (references key) (references value) | Lir.Spread value -> references value) |> Set.unionMany diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs index 1a5ae8b..f3ca83c 100644 --- a/src/FScript.Compiler/Hir.fs +++ b/src/FScript.Compiler/Hir.fs @@ -56,6 +56,7 @@ module Hir = | Match of Expression * MatchCase list | Range of Expression * Expression | RecordUpdate of Expression * (string * Expression) list + | RecordConvert of Expression * Type | IndexGet of Expression * Expression | Interpolate of InterpolationPart list | UnionCase of string * string * Expression option @@ -187,6 +188,7 @@ module Hir = | RecordUpdate(target, fields) -> lines.Add(typed "record-update"); expression (indent + 2) target fields |> List.iter (fun (_, item) -> expression (indent + 2) item) + | RecordConvert(target, targetType) -> lines.Add(typed $"record-convert {formatType targetType}"); expression (indent + 2) target | IndexGet(target, index) -> lines.Add(typed "index-get"); expression (indent + 2) target; expression (indent + 2) index | Interpolate parts -> lines.Add(typed "interpolate") diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index 5f26262..ea230f4 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -6,6 +6,7 @@ module HirLowering = type private State = { mutable NextSymbol: int ExpressionTypes: Map + Coercions: Map mutable UnionCases: Map mutable TypeDefinitions: Map mutable Diagnostics: CompilerDiagnostic list } @@ -158,7 +159,15 @@ module HirLowering = | TFun(value, _) -> value | _ -> Types.freshVar() match lowerExpression state environment fnType fn, lowerExpression state environment argumentType argument with - | Ok loweredFn, Ok loweredArgument -> Ok(make (Hir.Apply(loweredFn, loweredArgument))) + | Ok loweredFn, Ok loweredArgument -> + let argumentWithEvidence = + state.Coercions + |> Map.tryFind expression + |> Option.map (fun evidence -> + let targetType = representationType state evidence.TargetType + ({ loweredArgument with Type = targetType; Node = Hir.RecordConvert(loweredArgument, targetType) }: Hir.Expression)) + |> Option.defaultValue loweredArgument + Ok(make (Hir.Apply(loweredFn, argumentWithEvidence))) | Error left, Error right -> Error(left @ right) | Error errors, _ | _, Error errors -> Error errors @@ -319,9 +328,17 @@ module HirLowering = let lower (externs: CompilerExtern list) (resolved: TypeInfer.ResolvedProgram) : Result = let expressionTypes = - resolved.ExpressionTypes - |> List.fold (fun state item -> state |> Map.add item.Expr item.Type) Map.empty - let state = { NextSymbol = 0; ExpressionTypes = expressionTypes; UnionCases = Map.empty; TypeDefinitions = Map.empty; Diagnostics = [] } + let rec addExpression state (item: TypeInfer.ResolvedExpression) = + item.Children |> List.fold addExpression (Map.add item.Source item.Type state) + resolved.ResolvedStatements + |> List.collect _.Expressions + |> List.fold addExpression Map.empty + let coercions = + let rec collect state (item: TypeInfer.ResolvedExpression) = + let state = item.RecordCoercion |> Option.map (fun evidence -> Map.add item.Source evidence state) |> Option.defaultValue state + item.Children |> List.fold collect state + resolved.ResolvedStatements |> List.collect _.Expressions |> List.fold collect Map.empty + let state = { NextSymbol = 0; ExpressionTypes = expressionTypes; Coercions = coercions; UnionCases = Map.empty; TypeDefinitions = Map.empty; Diagnostics = [] } let builtins = BuiltinSignatures.builtinSchemes |> Map.toList diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index 056453f..fd5523d 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -82,6 +82,7 @@ module HirValidation = Set.union (references target) (cases |> List.map caseReferences |> Set.unionMany) | Hir.Range(startValue, endValue) | Hir.IndexGet(startValue, endValue) -> Set.union (references startValue) (references endValue) | Hir.RecordUpdate(target, fields) -> Set.union (references target) (fields |> List.map (snd >> references) |> Set.unionMany) + | Hir.RecordConvert(target, _) -> references target | Hir.Interpolate parts -> parts |> List.choose (function Hir.TextPart _ -> None | Hir.ValuePart item -> Some(references item)) |> Set.unionMany | Hir.UnionCase(_, _, payload) -> payload |> Option.map references |> Option.defaultValue Set.empty | Hir.Map entries -> diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index c8ac4ca..11c38ca 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -207,6 +207,9 @@ module IrArtifact = for name, field in fields do writer.WriteStartObject(); writer.WriteString("name", name); writer.WritePropertyName("value"); writeExpression writer field; writer.WriteEndObject() writer.WriteEndArray() + | Lir.RecordConvert(target, targetType) -> + writer.WriteString("kind", "record-convert"); writer.WritePropertyName("target"); writeExpression writer target + writer.WritePropertyName("targetType"); writeType writer targetType | Lir.IndexGet(target, index) -> writer.WriteString("kind", "index-get"); writer.WritePropertyName("target"); writeExpression writer target writer.WritePropertyName("index"); writeExpression writer index @@ -381,6 +384,7 @@ module IrArtifact = | "record-update" -> let fields = (property "fields" element).EnumerateArray() |> Seq.map (fun item -> stringProperty "name" item, readExpression (property "value" item)) |> Seq.toList Lir.RecordUpdate(readExpression (property "target" element), fields) + | "record-convert" -> Lir.RecordConvert(readExpression (property "target" element), readType (property "targetType" element)) | "index-get" -> Lir.IndexGet(readExpression (property "target" element), readExpression (property "index" element)) | "interpolate" -> Lir.Interpolate((property "parts" element).EnumerateArray() |> Seq.map (fun item -> diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index 6671d55..51f463c 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -44,6 +44,7 @@ module Lir = | DecisionTree of Expression * DecisionCase list | Range of Expression * Expression | RecordUpdate of Expression * (string * Expression) list + | RecordConvert of Expression * Type | IndexGet of Expression * Expression | Interpolate of InterpolationPart list | UnionCase of string * string * Expression option @@ -176,6 +177,7 @@ module LirLowering = Span = case.Span }: Lir.DecisionCase))) | Hir.Range(startValue, endValue) -> Lir.Range(expression startValue, expression endValue) | Hir.RecordUpdate(target, fields) -> Lir.RecordUpdate(expression target, fields |> List.map (fun (name, item) -> name, expression item)) + | Hir.RecordConvert(target, targetType) -> Lir.RecordConvert(expression target, targetType) | Hir.IndexGet(target, index) -> Lir.IndexGet(expression target, expression index) | Hir.Interpolate parts -> Lir.Interpolate(parts |> List.map (function Hir.TextPart text -> Lir.TextPart text | Hir.ValuePart item -> Lir.ValuePart(expression item))) | Hir.UnionCase(typeName, caseName, payload) -> Lir.UnionCase(typeName, caseName, Option.map expression payload) @@ -331,6 +333,7 @@ module LirValidation = | Lir.DecisionTree(target, cases) -> check target; cases |> List.iter (fun case -> let caseLocals = Set.union locals (patternBindings case.Pattern) in checkPattern target.Type case.Pattern; case.Guard |> Option.iter (fun guard -> checkExpression caseLocals guard; if not (compatible guard.Type TBool) then report "FSL2007" "LIR decision guard must be Boolean." guard.Span); checkExpression caseLocals case.Body; if not (compatible expression.Type case.Body.Type) then report "FSL2007" "LIR decision body type is inconsistent." case.Body.Span) | Lir.Range(first, last) -> check first; check last; if not (compatible first.Type TInt && compatible last.Type TInt && compatible expression.Type (TList TInt)) then report "FSL2004" "LIR range representation is inconsistent." expression.Span | Lir.RecordUpdate(target, fields) -> check target; fields |> List.iter (snd >> check); if not (compatible expression.Type target.Type) then report "FSL2004" "LIR record update representation is inconsistent." expression.Span + | Lir.RecordConvert(target, targetType) -> check target; match target.Type, targetType with TRecord _, (TNamed _ | TRecord _) when compatible expression.Type targetType -> () | TNamed _, TRecord _ when compatible expression.Type targetType -> () | _ -> report "FSL2004" "LIR record conversion evidence is inconsistent." expression.Span | Lir.IndexGet(target, index) -> check target; check index; match target.Type, expression.Type with TList item, TOption result when compatible index.Type TInt && compatible item result -> () | TMap(key, value), TOption result when compatible key index.Type && compatible value result -> () | _ -> report "FSL2004" "LIR index representation is inconsistent." expression.Span | Lir.Interpolate parts -> parts |> List.iter (function Lir.TextPart _ -> () | Lir.ValuePart value -> check value); if not (compatible expression.Type TString) then report "FSL2004" "LIR interpolation must produce string." expression.Span | Lir.UnionCase(typeName, caseName, payload) -> payload |> Option.iter check; match expression.Type with TUnion(actual, cases) when actual = typeName && Map.containsKey caseName cases -> () | _ -> report "FSL2004" "LIR union representation is inconsistent." expression.Span diff --git a/src/FScript.Interpreter.Lir/LirInterpreter.fs b/src/FScript.Interpreter.Lir/LirInterpreter.fs index 51a7776..03c245e 100644 --- a/src/FScript.Interpreter.Lir/LirInterpreter.fs +++ b/src/FScript.Interpreter.Lir/LirInterpreter.fs @@ -373,6 +373,11 @@ module LirInterpreter = |> collect |> Result.map (fun updates -> VRecord(List.fold (fun state (name, value) -> Map.add name value state) existing updates)) | Ok _ -> fail "FSI3015" expression.Span "Record update requires a record." | Error errors -> Error errors + | Lir.RecordConvert(target, _) -> + match evaluate environment target with + | Ok(VRecord _ as record) -> Ok record + | Ok _ -> fail "FSI3015" expression.Span "Record conversion requires a record." + | Error errors -> Error errors | Lir.IndexGet(target, index) -> match evaluate environment target, evaluate environment index with | Ok(VList values), Ok(VInt position) -> diff --git a/src/FScript.Language/TypeInfer.fs b/src/FScript.Language/TypeInfer.fs index c951321..964441e 100644 --- a/src/FScript.Language/TypeInfer.fs +++ b/src/FScript.Language/TypeInfer.fs @@ -187,6 +187,29 @@ module TypeInfer = Span: Span Scheme: Scheme option } + type ResolvedPrimitiveOperation = + | IntegerAddition | FloatingAddition + | IntegerSubtraction | FloatingSubtraction + | IntegerMultiplication | FloatingMultiplication + | IntegerDivision | FloatingDivision + | IntegerRemainder | StructuralEquality + | NumericComparison of string + | BooleanOperation of string + + type RecordCoercionEvidence = + { SourceType: Type + TargetType: Type + TargetName: string option } + + type ResolvedExpression = + { Source: Expr + Type: Type + Span: Span + Reference: ResolvedSymbolReference option + Primitive: ResolvedPrimitiveOperation option + RecordCoercion: RecordCoercionEvidence option + Children: ResolvedExpression list } + type private LocalVariableTypeCapture = { Name: string Span: Span @@ -200,8 +223,13 @@ module TypeInfer = | TSExpr of TypedExpr type TypedProgram = TypedStmt list + type ResolvedStatement = + { Source: TypedStmt + Expressions: ResolvedExpression list } + type ResolvedProgram = { TypedProgram: TypedProgram + ResolvedStatements: ResolvedStatement list ExpressionTypes: ResolvedExpressionType list Symbols: ResolvedSymbol list SymbolReferences: ResolvedSymbolReference list } @@ -1602,10 +1630,72 @@ module TypeInfer = | TSExpr expression -> visit globalEnvironment expression.Expr | TSBackendImport _ -> () | TSType _ -> () + let resolvedReferences = references |> Seq.toList + let finalType expression = expressionTypes |> Map.tryFind expression |> Option.defaultValue TUnit |> normalizeTypeVariables + let rec resolveExpression expression = + let expressionType = finalType expression + let children = + match expression with + | ELambda(_, body, _) | ESome(body, _) | EParen(body, _) | ERaise(body, _) -> [ body ] + | EApply(left, right, _) | EBinOp(_, left, right, _) | EAppend(left, right, _) | ECons(left, right, _) + | EIndexGet(left, right, _) | ERange(left, right, _) -> [ left; right ] + | EIf(condition, whenTrue, whenFalse, _) -> [ condition; whenTrue; whenFalse ] + | ELet(_, binding, body, _, _, _) | ELetPattern(_, binding, body, _) -> [ binding; body ] + | ELetRecGroup(bindings, body, _) -> + (bindings + |> List.map (fun (_, parameters, _, value, span) -> + Seq.foldBack (fun parameter inner -> ELambda(parameter, inner, span)) parameters value)) + @ [ body ] + | EFor(_, source, body, _) -> [ source; body ] + | EMatch(target, cases, _) -> target :: (cases |> List.collect (fun (_, guard, body, _) -> (Option.toList guard) @ [ body ])) + | EList(items, _) | ETuple(items, _) -> items + | ERecord(fields, _) | EStructuralRecord(fields, _) -> List.map snd fields + | EMap(entries, _) -> entries |> List.collect (function MEKeyValue(key, value) -> [ key; value ] | MESpread value -> [ value ]) + | ERecordUpdate(target, fields, _) | EStructuralRecordUpdate(target, fields, _) -> target :: List.map snd fields + | EFieldGet(target, _, _) -> [ target ] + | EInterpolatedString(parts, _) -> parts |> List.choose (function IPText _ -> None | IPExpr item -> Some item) + | EUnit _ | ELiteral _ | EVar _ | ENone _ | ETypeOf _ | ENameOf _ -> [] + |> List.map resolveExpression + let reference = + let span = Ast.spanOfExpr expression + resolvedReferences |> List.tryFind (fun item -> item.Span = span) + let primitive = + match expression with + | EBinOp(op, left, _, _) -> + match op, expressionType with + | "+", TInt -> Some IntegerAddition | "+", TFloat -> Some FloatingAddition + | "-", TInt -> Some IntegerSubtraction | "-", TFloat -> Some FloatingSubtraction + | "*", TInt -> Some IntegerMultiplication | "*", TFloat -> Some FloatingMultiplication + | "/", TInt -> Some IntegerDivision | "/", TFloat -> Some FloatingDivision + | "%", TInt -> Some IntegerRemainder | "=", _ -> Some StructuralEquality + | ("<" | ">" | "<=" | ">="), _ -> Some(NumericComparison op) + | ("&&" | "||"), _ -> Some(BooleanOperation op) + | _ -> None + | _ -> None + let coercion = + match expression with + | EApply(fn, argument, _) -> + match finalType fn, finalType argument with + | TFun((TNamed target as targetType), _), (TRecord _ as sourceType) -> Some { SourceType = sourceType; TargetType = targetType; TargetName = Some target } + | TFun((TRecord _ as targetType), _), (TNamed source as sourceType) -> Some { SourceType = sourceType; TargetType = targetType; TargetName = Some source } + | _ -> None + | _ -> None + { Source = expression; Type = expressionType; Span = Ast.spanOfExpr expression; Reference = reference + Primitive = primitive; RecordCoercion = coercion; Children = children } + let resolvedStatements = + typed |> List.map (fun statement -> + let roots = + match statement with + | TSLet(_, expression, _, _, _, _) | TSLetPattern(_, expression, _, _, _) -> [ expression ] + | TSLetRecGroup(bindings, _, _) -> bindings |> List.map (fun (_, expression, _, _) -> expression) + | TSExpr expression -> [ expression.Expr ] + | TSBackendImport _ | TSType _ -> [] + { Source = statement; Expressions = List.map resolveExpression roots }) { TypedProgram = typed + ResolvedStatements = resolvedStatements ExpressionTypes = expressions Symbols = symbols |> Seq.toList - SymbolReferences = references |> Seq.toList } + SymbolReferences = resolvedReferences } let inferResolvedProgram (program: Program) : ResolvedProgram = inferResolvedProgramWithExterns [] program diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index 94bf87a..736b7a7 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -99,3 +99,13 @@ type IrArtifactTests() = match IrArtifact.deserialize bytes with | Ok _ -> () | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``IR artifact preserves explicit structural record conversions`` () = + let source = "type Person = { Name: string; Age: int }\nlet age (person: Person) = person.Age\nage {| Name = \"Ada\"; Age = 42 |}" + let lir = (compile source).Lir + let bytes = IrArtifact.serialize lir + Assert.That(Encoding.UTF8.GetString(bytes), Does.Contain("\"kind\": \"record-convert\"")) + match IrArtifact.deserialize bytes with + | Ok loaded -> Assert.That(loaded, Is.EqualTo(lir)) + | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) diff --git a/tests/FScript.Language.Tests/ResolvedInferenceTests.fs b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs index 31ff6bd..9babaf9 100644 --- a/tests/FScript.Language.Tests/ResolvedInferenceTests.fs +++ b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs @@ -5,6 +5,8 @@ open NUnit.Framework [] type ResolvedInferenceTests() = + let rec flatten (expression: TypeInfer.ResolvedExpression) = expression :: List.collect flatten expression.Children + [] member _.``Resolved inference captures every expression without changing legacy output`` () = let program = Helpers.parse "let add x y = x + y\nadd 1 2" @@ -51,3 +53,24 @@ type ResolvedInferenceTests() = |> List.find (fun symbol -> symbol.Name = "identity" && symbol.Kind = TypeInfer.DeclarationSymbol) |> _.Scheme Assert.That(scheme, Is.EqualTo(Some(Forall([ 0 ], TFun(TVar 0, TVar 0))))) + + [] + member _.``Resolved frontend exposes a recursively typed operation tree`` () = + let resolved = Helpers.parse "let add x y = x + y\nadd 1 2" |> TypeInfer.inferResolvedProgram + let expressions = resolved.ResolvedStatements |> List.collect _.Expressions |> List.collect flatten + Assert.That(expressions, Is.Not.Empty) + Assert.That(expressions |> List.forall (fun expression -> expression.Type <> TVar -1), Is.True) + Assert.That(expressions |> List.exists (fun expression -> expression.Primitive = Some TypeInfer.IntegerAddition), Is.True) + Assert.That(expressions |> List.exists (fun expression -> expression.Reference |> Option.exists (fun reference -> reference.Name = "add")), Is.True) + + [] + member _.``Resolved applications retain record coercion evidence`` () = + let source = "type Person = { Name: string; Age: int }\nlet age (person: Person) = person.Age\nage {| Name = \"Ada\"; Age = 42 |}" + let resolved = Helpers.parse source |> TypeInfer.inferResolvedProgram + let expressions = resolved.ResolvedStatements |> List.collect _.Expressions |> List.collect flatten + let coercion = expressions |> List.choose _.RecordCoercion |> List.tryHead + match coercion with + | Some evidence -> + Assert.That(evidence.TargetName, Is.EqualTo(Some "Person")) + match evidence.SourceType with TRecord fields -> Assert.That(fields.Count, Is.EqualTo(2)) | _ -> Assert.Fail("Expected structural record source") + | None -> Assert.Fail("Expected structural-to-named record coercion evidence") From dbacb8b37f2e92fcabdaee0eb59f561df2fe1bae Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 19:01:59 +0200 Subject: [PATCH 038/133] feat(compiler): record explicit closure captures --- CHANGELOG.md | 2 + docs/architecture/typed-ir-and-backends.md | 2 +- docs/specs/compiler-semantics-baseline.md | 2 +- src/FScript.Backend.Go/GoBackend.fs | 4 +- src/FScript.Compiler/IrArtifact.fs | 12 +++- src/FScript.Compiler/Lir.fs | 61 ++++++++++++++++++- src/FScript.Interpreter.Lir/LirInterpreter.fs | 4 +- .../FScript.Compiler.Tests/IrArtifactTests.fs | 20 ++++++ 8 files changed, 96 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index cc41431..52e490c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added deterministic explicit closure captures to LIR serialization, validation, and interpretation. + - Exposed recursively typed resolved expressions and preserved explicit structural record conversions through HIR, LIR, and IR artifacts. - Added imported record declarations with serialized HIR/LIR metadata, primitive Go boundary adapters, and explicit unsupported-shape diagnostics. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 260ebf8..6cd7dff 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -22,7 +22,7 @@ Validated LIR consumers are separate leaf assemblies. `FScript.Interpreter.Lir` HIR is language-oriented and contains resolved artifact-local symbol identities, complete semantic types and schemes, source locations, declaration attributes, and explicit nodes for named/structural record conversions. Conversion evidence is produced by inference, not rediscovered from backend representations. HIR contains no Go types. -LIR is executable but not a Go AST. It uses lifted functions, explicit captures and closure creation, resolved primitive operations, aggregate construction/projection, and structured control flow. Pattern matching becomes deterministic tests and projections before reaching a backend. Semantic types may remain alongside selected runtime representations for validation and diagnostics. +LIR is executable but not a Go AST. Closure creation carries a canonical ordered set of captured symbol IDs, alongside resolved primitive operations, aggregate construction/projection, and structured control flow. Validation recomputes each closure's free-symbol set and rejects forged, missing, duplicate, or unordered captures. Pattern matching becomes deterministic tests and projections before reaching a backend. Semantic types may remain alongside selected runtime representations for validation and diagnostics; moving closure bodies into a separate function table remains a representation-only lowering step. The pass pipeline is HIR lowering, HIR validation, LIR lowering, `LirOptimization.optimize`, then LIR validation. The optimization pass is intentionally identity-only in version 1; its explicit boundary prevents later transformations from being hidden inside lowering or a backend. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index c046545..337c061 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -20,7 +20,7 @@ Ranges preserve inclusive ascending or descending order. List indexing returns a Declared unions retain their type and case identity plus an optional erased payload. Constructors are deterministic compiler declarations, qualified and unqualified case references share one identity, pattern tests check both type and case tags, and equality recursively compares payloads. Compiled maps use an ordered entry representation for string and integer keys. Explicit key/value entries replace an earlier value; spread entries retain values already present on their left. Indexing returns an option, and deterministic ordering participates in artifacts, equality, formatting, and generated Go behavior. -- Functions are curried. Multi-argument declarations are nested lambdas, application is one argument at a time, and closures capture their lexical environment. Single and mutual recursive function groups are supported. +- Functions are curried. Multi-argument declarations are nested lambdas, application is one argument at a time, and LIR closures record only their deterministic free-symbol capture set. Single and mutual recursive function groups are supported. - Local mutual-recursive function groups initialize atomically: every peer is visible from every closure before the group body executes. ## Types and values diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index 109d074..aeda9d6 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -245,7 +245,7 @@ module GoBackend = environment |> Map.tryFind id |> Option.map Ok |> Option.defaultValue (Error(sprintf "Unknown symbol %%%d" (Lir.symbolNumber id))) - | Lir.MakeClosure(parameter, _, parameterType, body) -> + | Lir.MakeClosure(parameter, _, parameterType, _, body) -> let name = identifier "arg" parameter let parameterValue = cast parameterType "rawValue" emitExpression (environment |> Map.add parameter name) body @@ -568,7 +568,7 @@ func Environment() map[string]any { let many values = values |> List.map references |> Set.unionMany match expression.Node with | Lir.Load id -> Set.singleton id - | Lir.MakeClosure(_, _, _, body) | Lir.Fail body | Lir.FieldGet(body, _) | Lir.SomeValue body -> references body + | Lir.MakeClosure(_, _, _, _, body) | Lir.Fail body | Lir.FieldGet(body, _) | Lir.SomeValue body -> references body | Lir.CallClosure(left, right) | Lir.Append(left, right) | Lir.Cons(left, right) | Lir.Range(left, right) | Lir.IndexGet(left, right) -> Set.union (references left) (references right) | Lir.Branch(condition, whenTrue, whenFalse) -> many [ condition; whenTrue; whenFalse ] | Lir.Operation(_, values) | Lir.Tuple values | Lir.List values -> many values diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index 11c38ca..d6e75cf 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -143,9 +143,10 @@ module IrArtifact = | Lir.Constant constant -> writer.WriteString("kind", "constant"); writeConstantValue writer constant | Lir.Load id -> writer.WriteString("kind", "load"); writer.WriteNumber("symbol", Lir.symbolNumber id) - | Lir.MakeClosure(id, name, parameterType, body) -> + | Lir.MakeClosure(id, name, parameterType, captures, body) -> writer.WriteString("kind", "closure"); writer.WriteNumber("symbol", Lir.symbolNumber id); writer.WriteString("name", name) writer.WritePropertyName("parameterType"); writeType writer parameterType + writer.WriteStartArray("captures"); captures |> List.iter (Lir.symbolNumber >> writer.WriteNumberValue); writer.WriteEndArray() writer.WritePropertyName("body"); writeExpression writer body | Lir.CallClosure(fn, argument) -> writer.WriteString("kind", "call"); writer.WritePropertyName("function"); writeExpression writer fn @@ -350,8 +351,13 @@ module IrArtifact = match stringProperty "kind" element with | "constant" -> Lir.Constant(readConstant element) | "load" -> Lir.Load(Lir.SymbolId((property "symbol" element).GetInt32())) - | "closure" -> Lir.MakeClosure(Lir.SymbolId((property "symbol" element).GetInt32()), stringProperty "name" element, - readType (property "parameterType" element), readExpression (property "body" element)) + | "closure" -> + Lir.MakeClosure( + Lir.SymbolId((property "symbol" element).GetInt32()), + stringProperty "name" element, + readType (property "parameterType" element), + (property "captures" element).EnumerateArray() |> Seq.map (fun item -> Lir.SymbolId(item.GetInt32())) |> Seq.toList, + readExpression (property "body" element)) | "call" -> Lir.CallClosure(readExpression (property "function" element), readExpression (property "argument" element)) | "branch" -> Lir.Branch(readExpression (property "condition" element), readExpression (property "whenTrue" element), readExpression (property "whenFalse" element)) | "operation" -> Lir.Operation(readOperation (stringProperty "operation" element), (property "arguments" element).EnumerateArray() |> Seq.map readExpression |> Seq.toList) diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index 51f463c..4660aae 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -26,7 +26,7 @@ module Lir = and ExpressionNode = | Constant of Constant | Load of SymbolId - | MakeClosure of SymbolId * string * Type * Expression + | MakeClosure of SymbolId * string * Type * SymbolId list * Expression | CallClosure of Expression * Expression | Branch of Expression * Expression * Expression | Operation of Operation * Expression list @@ -148,12 +148,62 @@ module LirLowering = | Hir.TypePattern patternType -> Lir.TypePattern patternType { Type = value.Type; Span = value.Span; Node = node } + let rec private patternBindings (value: Lir.Pattern) = + match value.Node with + | Lir.Binding(id, _) -> Set.singleton id + | Lir.ConsPattern(head, tail) -> Set.union (patternBindings head) (patternBindings tail) + | Lir.TuplePattern items -> items |> List.map patternBindings |> Set.unionMany + | Lir.RecordPattern fields -> fields |> List.map (snd >> patternBindings) |> Set.unionMany + | Lir.MapPattern(clauses, tail) -> + [ yield! clauses |> List.collect (fun (key, value) -> [ patternBindings key; patternBindings value ]) + tail |> Option.map patternBindings |> Option.defaultValue Set.empty ] + |> Set.unionMany + | Lir.SomePattern item -> patternBindings item + | Lir.UnionPattern(_, _, payload) -> payload |> Option.map patternBindings |> Option.defaultValue Set.empty + | _ -> Set.empty + + let rec capturedReferences bound (value: Lir.Expression) = + let free = capturedReferences bound + match value.Node with + | Lir.Load id -> if Set.contains id bound then Set.empty else Set.singleton id + | Lir.MakeClosure(_, _, _, captures, _) -> Set.difference (Set.ofList captures) bound + | Lir.CallClosure(fn, argument) | Lir.Append(fn, argument) | Lir.Cons(fn, argument) + | Lir.Range(fn, argument) | Lir.IndexGet(fn, argument) -> Set.union (free fn) (free argument) + | Lir.Branch(condition, whenTrue, whenFalse) -> [ free condition; free whenTrue; free whenFalse ] |> Set.unionMany + | Lir.Operation(_, arguments) | Lir.Tuple arguments | Lir.List arguments -> arguments |> List.map free |> Set.unionMany + | Lir.Record fields -> fields |> List.map (snd >> free) |> Set.unionMany + | Lir.FieldGet(target, _) | Lir.SomeValue target | Lir.Fail target | Lir.RecordConvert(target, _) -> free target + | Lir.Let(id, _, recursive, binding, body) -> + let scope = Set.add id bound + Set.union (capturedReferences (if recursive then scope else bound) binding) (capturedReferences scope body) + | Lir.LetRecursiveGroup(bindings, body) -> + let scope = bindings |> List.fold (fun state (id, _, _, _) -> Set.add id state) bound + (capturedReferences scope body :: (bindings |> List.map (fun (_, _, _, binding) -> capturedReferences scope binding))) |> Set.unionMany + | Lir.DecisionTree(target, cases) -> + free target + :: (cases |> List.collect (fun case -> + let scope = Set.union bound (patternBindings case.Pattern) + [ yield! case.Guard |> Option.map (capturedReferences scope) |> Option.toList + capturedReferences scope case.Body ])) + |> Set.unionMany + | Lir.RecordUpdate(target, fields) -> free target :: (fields |> List.map (snd >> free)) |> Set.unionMany + | Lir.Interpolate parts -> parts |> List.choose (function Lir.TextPart _ -> None | Lir.ValuePart item -> Some(free item)) |> Set.unionMany + | Lir.UnionCase(_, _, payload) -> payload |> Option.map free |> Option.defaultValue Set.empty + | Lir.Map entries -> entries |> List.map (function Lir.KeyValue(key, item) -> Set.union (free key) (free item) | Lir.Spread item -> free item) |> Set.unionMany + | Lir.ForEach(id, _, _, source, body) -> Set.union (free source) (capturedReferences (Set.add id bound) body) + | Lir.Constant _ | Lir.NoneValue | Lir.Intrinsic _ | Lir.HostExternal _ | Lir.RuntimeContext + | Lir.TargetImport _ | Lir.TypeToken _ -> Set.empty + let rec private expression (value: Hir.Expression) : Lir.Expression = let node = match value.Node with | Hir.Constant value -> Lir.Constant(constant value) | Hir.Reference id -> Lir.Load(symbol id) - | Hir.Lambda(id, name, parameterType, body) -> Lir.MakeClosure(symbol id, name, parameterType, expression body) + | Hir.Lambda(id, name, parameterType, body) -> + let parameter = symbol id + let loweredBody = expression body + let captures = capturedReferences (Set.singleton parameter) loweredBody |> Set.toList |> List.sortBy Lir.symbolNumber + Lir.MakeClosure(parameter, name, parameterType, captures, loweredBody) | Hir.Apply(fn, argument) -> Lir.CallClosure(expression fn, expression argument) | Hir.If(condition, whenTrue, whenFalse) -> Lir.Branch(expression condition, expression whenTrue, expression whenFalse) | Hir.Primitive(op, arguments) -> Lir.Operation(operation op, arguments |> List.map expression) @@ -293,7 +343,12 @@ module LirValidation = match expression.Node with | Lir.Constant value when not (compatible expression.Type (literalType value)) -> report "FSL2004" "LIR constant has an inconsistent type." expression.Span | Lir.Load id when not (Set.contains id locals || Set.contains id declared) -> report "FSL2005" (sprintf "LIR reference %%%d is unresolved." (Lir.symbolNumber id)) expression.Span - | Lir.MakeClosure(parameter, _, parameterType, body) -> + | Lir.MakeClosure(parameter, _, parameterType, captures, body) -> + let expectedCaptures = LirLowering.capturedReferences (Set.singleton parameter) body |> Set.toList |> List.sortBy Lir.symbolNumber + if captures <> expectedCaptures then + report "FSL2013" "LIR closure captures do not match its ordered free-symbol set." expression.Span + if captures |> List.exists (fun id -> not (Set.contains id locals || Set.contains id declared)) then + report "FSL2005" "LIR closure contains an unresolved capture." expression.Span checkExpression (Set.add parameter locals) body match expression.Type with TFun(argument, result) when compatible argument parameterType && compatible result body.Type -> () | _ -> report "FSL2006" "LIR closure signature is inconsistent with its body." expression.Span | Lir.CallClosure(fn, argument) -> diff --git a/src/FScript.Interpreter.Lir/LirInterpreter.fs b/src/FScript.Interpreter.Lir/LirInterpreter.fs index 03c245e..93146b3 100644 --- a/src/FScript.Interpreter.Lir/LirInterpreter.fs +++ b/src/FScript.Interpreter.Lir/LirInterpreter.fs @@ -260,7 +260,9 @@ module LirInterpreter = | Lir.Load id -> environment |> Map.tryFind id |> Option.map Ok |> Option.defaultWith (fun () -> fail "FSI3001" expression.Span (sprintf "Unknown LIR symbol %%%d." (Lir.symbolNumber id))) - | Lir.MakeClosure(id, name, _, body) -> Ok(VClosure(id, name, body, ref environment)) + | Lir.MakeClosure(id, name, _, captures, body) -> + let captured = captures |> List.choose (fun symbol -> Map.tryFind symbol environment |> Option.map (fun value -> symbol, value)) |> Map.ofList + Ok(VClosure(id, name, body, ref captured)) | Lir.CallClosure(fn, argument) -> match evaluate environment fn, evaluate environment argument with | Ok fnValue, Ok value -> applyValue expression.Span fnValue value diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index 736b7a7..b8acbe3 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -109,3 +109,23 @@ type IrArtifactTests() = match IrArtifact.deserialize bytes with | Ok loaded -> Assert.That(loaded, Is.EqualTo(lir)) | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``LIR closures serialize deterministic explicit captures`` () = + let source = "let outer x = (let inner y = x + y\n inner\n)\nouter 40 2" + let bytes = (compile source).Lir |> IrArtifact.serialize + let json = Encoding.UTF8.GetString(bytes) + Assert.That(json, Does.Match("\\\"captures\\\": \\[\\s+[0-9]+")) + match IrArtifact.deserialize bytes with + | Ok _ -> () + | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) + + [] + member _.``IR artifact validation rejects forged closure captures`` () = + let source = "let outer x = (let inner y = x + y\n inner\n)\nouter 40 2" + let json = (compile source).Lir |> IrArtifact.serialize |> Encoding.UTF8.GetString + let capturePattern = System.Text.RegularExpressions.Regex("\\\"captures\\\": \\[\\s+[0-9]+\\s+\\]") + let invalid = capturePattern.Replace(json, "\"captures\": [ ]", 1) |> Encoding.UTF8.GetBytes + match IrArtifact.deserialize invalid with + | Ok _ -> Assert.Fail("Expected invalid closure-capture diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSL2013")) From e8d62a301f0ceda215f80b83414e2532abc778f1 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 19:10:11 +0200 Subject: [PATCH 039/133] feat(compiler): lift LIR function bodies --- CHANGELOG.md | 2 + docs/architecture/typed-ir-and-backends.md | 2 +- src/FScript.Backend.Go/GoBackend.fs | 23 ++-- src/FScript.Compiler/IrArtifact.fs | 33 +++-- src/FScript.Compiler/Lir.fs | 126 ++++++++++++------ src/FScript.Interpreter.Lir/LirInterpreter.fs | 9 +- tests/FScript.Compiler.Tests/HirTests.fs | 6 +- .../FScript.Compiler.Tests/IrArtifactTests.fs | 5 +- 8 files changed, 140 insertions(+), 66 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 52e490c..a041bb2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Lifted LIR lambda bodies into a deterministic validated function table consumed by artifacts, interpretation, and Go generation. + - Added deterministic explicit closure captures to LIR serialization, validation, and interpretation. - Exposed recursively typed resolved expressions and preserved explicit structural record conversions through HIR, LIR, and IR artifacts. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 6cd7dff..897d848 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -22,7 +22,7 @@ Validated LIR consumers are separate leaf assemblies. `FScript.Interpreter.Lir` HIR is language-oriented and contains resolved artifact-local symbol identities, complete semantic types and schemes, source locations, declaration attributes, and explicit nodes for named/structural record conversions. Conversion evidence is produced by inference, not rediscovered from backend representations. HIR contains no Go types. -LIR is executable but not a Go AST. Closure creation carries a canonical ordered set of captured symbol IDs, alongside resolved primitive operations, aggregate construction/projection, and structured control flow. Validation recomputes each closure's free-symbol set and rejects forged, missing, duplicate, or unordered captures. Pattern matching becomes deterministic tests and projections before reaching a backend. Semantic types may remain alongside selected runtime representations for validation and diagnostics; moving closure bodies into a separate function table remains a representation-only lowering step. +LIR is executable but not a Go AST. Lambda bodies are lifted into a deterministic program-level function table. Closure creation references a function ID and carries its canonical ordered set of captured symbol IDs, alongside resolved primitive operations, aggregate construction/projection, and structured control flow. Validation recomputes each lifted function's free-symbol set and rejects unknown function IDs or forged, missing, duplicate, or unordered captures. Pattern matching becomes deterministic tests and projections before reaching a backend. Semantic types remain alongside selected runtime representations for validation and diagnostics. The pass pipeline is HIR lowering, HIR validation, LIR lowering, `LirOptimization.optimize`, then LIR validation. The optimization pass is intentionally identity-only in version 1; its explicit boundary prevents later transformations from being hidden inside lowering or a backend. diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index aeda9d6..d090b2c 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -11,6 +11,7 @@ type GeneratedFile = { RelativePath: string; Content: string } module GoBackend = let private currentImportedRecords = AsyncLocal option>() + let private currentFunctions = AsyncLocal>() let private tryImportedRecord name = currentImportedRecords.Value |> Option.bind (Map.tryFind name) let private indent (value: string) = value.Replace("\n", "\n\t") @@ -245,13 +246,16 @@ module GoBackend = environment |> Map.tryFind id |> Option.map Ok |> Option.defaultValue (Error(sprintf "Unknown symbol %%%d" (Lir.symbolNumber id))) - | Lir.MakeClosure(parameter, _, parameterType, _, body) -> - let name = identifier "arg" parameter - let parameterValue = cast parameterType "rawValue" - emitExpression (environment |> Map.add parameter name) body - |> Result.map (fun bodyCode -> - let indentedBody = bodyCode.Replace("\n", "\n\t") - $"fsruntime.Closure{{Call: func(rawValue any) any {{\n\t{name} := {parameterValue}\n\t_ = {name}\n\treturn {indentedBody}\n}}}}") + | Lir.MakeClosure(functionId, _) -> + match currentFunctions.Value |> Map.tryFind functionId with + | None -> Error $"Unknown lifted function {Lir.functionNumber functionId}" + | Some fn -> + let name = identifier "arg" fn.Parameter + let parameterValue = cast fn.ParameterType "rawValue" + emitExpression (environment |> Map.add fn.Parameter name) fn.Body + |> Result.map (fun bodyCode -> + let indentedBody = bodyCode.Replace("\n", "\n\t") + $"fsruntime.Closure{{Call: func(rawValue any) any {{\n\t{name} := {parameterValue}\n\t_ = {name}\n\treturn {indentedBody}\n}}}}") | Lir.CallClosure(fn, argument) -> match emitExpression environment fn, emitExpression environment argument with | Ok fnCode, Ok argumentCode -> Ok(cast expression.Type $"fsruntime.Call({fnCode}, {argumentCode})") @@ -495,6 +499,7 @@ func Environment() map[string]any { let private generateCore options (program: Lir.Program) : Result = try currentImportedRecords.Value <- Some(program.ImportedRecords |> List.map (fun record -> record.Name, record) |> Map.ofList) + currentFunctions.Value <- program.Functions |> List.map (fun fn -> fn.Id, fn) |> Map.ofList let targetImports = program.Declarations |> List.choose (fun declaration -> match declaration.Value.Node with Lir.TargetImport(modulePath, memberName) -> Some(modulePath, memberName, declaration.Type, declaration.Span) | _ -> None) @@ -556,6 +561,7 @@ func Environment() map[string]any { let generate options (program: Lir.Program) : Result = currentImportedRecords.Value <- Some(program.ImportedRecords |> List.map (fun record -> record.Name, record) |> Map.ofList) + currentFunctions.Value <- program.Functions |> List.map (fun fn -> fn.Id, fn) |> Map.ofList let rec containsTask = function | TTask _ -> true | TFun(argument, result) | TMap(argument, result) -> containsTask argument || containsTask result @@ -568,7 +574,8 @@ func Environment() map[string]any { let many values = values |> List.map references |> Set.unionMany match expression.Node with | Lir.Load id -> Set.singleton id - | Lir.MakeClosure(_, _, _, _, body) | Lir.Fail body | Lir.FieldGet(body, _) | Lir.SomeValue body -> references body + | Lir.MakeClosure(_, captures) -> Set.ofList captures + | Lir.Fail body | Lir.FieldGet(body, _) | Lir.SomeValue body -> references body | Lir.CallClosure(left, right) | Lir.Append(left, right) | Lir.Cons(left, right) | Lir.Range(left, right) | Lir.IndexGet(left, right) -> Set.union (references left) (references right) | Lir.Branch(condition, whenTrue, whenFalse) -> many [ condition; whenTrue; whenFalse ] | Lir.Operation(_, values) | Lir.Tuple values | Lir.List values -> many values diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index d6e75cf..5c441af 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -20,6 +20,7 @@ module IrArtifact = let private sourceMetadata (program: Lir.Program) = let paths = [ yield! program.ImportedRecords |> List.choose (fun declaration -> declaration.Span.Start.File) + yield! program.Functions |> List.choose (fun fn -> fn.Span.Start.File) yield! program.Declarations |> List.choose (fun declaration -> declaration.Span.Start.File) match program.Entry with Some entry -> yield! entry.Span.Start.File |> Option.toList | None -> () ] |> List.distinct @@ -143,11 +144,9 @@ module IrArtifact = | Lir.Constant constant -> writer.WriteString("kind", "constant"); writeConstantValue writer constant | Lir.Load id -> writer.WriteString("kind", "load"); writer.WriteNumber("symbol", Lir.symbolNumber id) - | Lir.MakeClosure(id, name, parameterType, captures, body) -> - writer.WriteString("kind", "closure"); writer.WriteNumber("symbol", Lir.symbolNumber id); writer.WriteString("name", name) - writer.WritePropertyName("parameterType"); writeType writer parameterType + | Lir.MakeClosure(id, captures) -> + writer.WriteString("kind", "closure"); writer.WriteNumber("function", Lir.functionNumber id) writer.WriteStartArray("captures"); captures |> List.iter (Lir.symbolNumber >> writer.WriteNumberValue); writer.WriteEndArray() - writer.WritePropertyName("body"); writeExpression writer body | Lir.CallClosure(fn, argument) -> writer.WriteString("kind", "call"); writer.WritePropertyName("function"); writeExpression writer fn writer.WritePropertyName("argument"); writeExpression writer argument @@ -265,6 +264,15 @@ module IrArtifact = for name, fieldType in record.Fields do writer.WriteStartObject(); writer.WriteString("name", name); writer.WritePropertyName("type"); writeType writer fieldType; writer.WriteEndObject() writer.WriteEndArray(); writer.WritePropertyName("span"); writeSpan writer record.Span; writer.WriteEndObject() + writer.WriteEndArray(); writer.WriteStartArray("functions") + for fn in program.Functions do + writer.WriteStartObject(); writer.WriteNumber("id", Lir.functionNumber fn.Id) + writer.WriteNumber("parameter", Lir.symbolNumber fn.Parameter); writer.WriteString("name", fn.Name) + writer.WritePropertyName("parameterType"); writeType writer fn.ParameterType + writer.WriteStartArray("captures"); fn.Captures |> List.iter (Lir.symbolNumber >> writer.WriteNumberValue); writer.WriteEndArray() + writer.WritePropertyName("returnType"); writeType writer fn.ReturnType + writer.WritePropertyName("span"); writeSpan writer fn.Span + writer.WritePropertyName("body"); writeExpression writer fn.Body; writer.WriteEndObject() writer.WriteEndArray(); writer.WriteStartArray("declarations") for declaration in program.Declarations do writer.WriteStartObject(); writer.WriteNumber("id", Lir.symbolNumber declaration.Id); writer.WriteString("name", declaration.Name) @@ -353,11 +361,8 @@ module IrArtifact = | "load" -> Lir.Load(Lir.SymbolId((property "symbol" element).GetInt32())) | "closure" -> Lir.MakeClosure( - Lir.SymbolId((property "symbol" element).GetInt32()), - stringProperty "name" element, - readType (property "parameterType" element), - (property "captures" element).EnumerateArray() |> Seq.map (fun item -> Lir.SymbolId(item.GetInt32())) |> Seq.toList, - readExpression (property "body" element)) + Lir.FunctionId((property "function" element).GetInt32()), + (property "captures" element).EnumerateArray() |> Seq.map (fun item -> Lir.SymbolId(item.GetInt32())) |> Seq.toList) | "call" -> Lir.CallClosure(readExpression (property "function" element), readExpression (property "argument" element)) | "branch" -> Lir.Branch(readExpression (property "condition" element), readExpression (property "whenTrue" element), readExpression (property "whenFalse" element)) | "operation" -> Lir.Operation(readOperation (stringProperty "operation" element), (property "arguments" element).EnumerateArray() |> Seq.map readExpression |> Seq.toList) @@ -447,9 +452,19 @@ module IrArtifact = ({ Id = Lir.SymbolId((property "id" item).GetInt32()); Name = stringProperty "name" item; Type = readType (property "type" item) IsRecursive = (property "recursive" item).GetBoolean(); IsExported = (property "exported" item).GetBoolean() Span = readSpan (property "span" item); Value = readExpression (property "value" item) }: Lir.Declaration)) |> Seq.toList + let functions = (property "functions" programElement).EnumerateArray() |> Seq.map (fun (item: JsonElement) -> + ({ Id = Lir.FunctionId((property "id" item).GetInt32()) + Parameter = Lir.SymbolId((property "parameter" item).GetInt32()) + Name = stringProperty "name" item + ParameterType = readType (property "parameterType" item) + Captures = (property "captures" item).EnumerateArray() |> Seq.map (fun capture -> Lir.SymbolId(capture.GetInt32())) |> Seq.toList + ReturnType = readType (property "returnType" item) + Span = readSpan (property "span" item) + Body = readExpression (property "body" item) }: Lir.Function)) |> Seq.toList let entryElement = property "entry" programElement let program : Lir.Program = { ImportedRecords = importedRecords + Functions = functions Declarations = declarations Entry = if entryElement.ValueKind = JsonValueKind.Null then None else Some(readExpression entryElement) } LirValidation.validate program diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index 4660aae..d6b975a 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -4,6 +4,7 @@ open FScript.Language module Lir = type SymbolId = SymbolId of int + type FunctionId = FunctionId of int type Constant = | Unit @@ -26,7 +27,7 @@ module Lir = and ExpressionNode = | Constant of Constant | Load of SymbolId - | MakeClosure of SymbolId * string * Type * SymbolId list * Expression + | MakeClosure of FunctionId * SymbolId list | CallClosure of Expression * Expression | Branch of Expression * Expression * Expression | Operation of Operation * Expression list @@ -98,6 +99,16 @@ module Lir = Span: Span Value: Expression } + type Function = + { Id: FunctionId + Parameter: SymbolId + Name: string + ParameterType: Type + Captures: SymbolId list + ReturnType: Type + Span: Span + Body: Expression } + type ImportedRecord = { Name: string ModulePath: string @@ -108,12 +119,23 @@ module Lir = type Program = { ImportedRecords: ImportedRecord list + Functions: Function list Declarations: Declaration list Entry: Expression option } let symbolNumber (SymbolId value) = value + let functionNumber (FunctionId value) = value module LirLowering = + type private State = + { mutable NextFunction: int + Functions: ResizeArray } + + let private newFunction state = + let id = Lir.FunctionId state.NextFunction + state.NextFunction <- state.NextFunction + 1 + id + let private symbol (Hir.SymbolId value) = Lir.SymbolId value let private operation = function @@ -166,7 +188,7 @@ module LirLowering = let free = capturedReferences bound match value.Node with | Lir.Load id -> if Set.contains id bound then Set.empty else Set.singleton id - | Lir.MakeClosure(_, _, _, captures, _) -> Set.difference (Set.ofList captures) bound + | Lir.MakeClosure(_, captures) -> Set.difference (Set.ofList captures) bound | Lir.CallClosure(fn, argument) | Lir.Append(fn, argument) | Lir.Cons(fn, argument) | Lir.Range(fn, argument) | Lir.IndexGet(fn, argument) -> Set.union (free fn) (free argument) | Lir.Branch(condition, whenTrue, whenFalse) -> [ free condition; free whenTrue; free whenFalse ] |> Set.unionMany @@ -194,65 +216,73 @@ module LirLowering = | Lir.Constant _ | Lir.NoneValue | Lir.Intrinsic _ | Lir.HostExternal _ | Lir.RuntimeContext | Lir.TargetImport _ | Lir.TypeToken _ -> Set.empty - let rec private expression (value: Hir.Expression) : Lir.Expression = + let rec private expression state (value: Hir.Expression) : Lir.Expression = let node = match value.Node with | Hir.Constant value -> Lir.Constant(constant value) | Hir.Reference id -> Lir.Load(symbol id) | Hir.Lambda(id, name, parameterType, body) -> let parameter = symbol id - let loweredBody = expression body + let functionId = newFunction state + let loweredBody = expression state body let captures = capturedReferences (Set.singleton parameter) loweredBody |> Set.toList |> List.sortBy Lir.symbolNumber - Lir.MakeClosure(parameter, name, parameterType, captures, loweredBody) - | Hir.Apply(fn, argument) -> Lir.CallClosure(expression fn, expression argument) - | Hir.If(condition, whenTrue, whenFalse) -> Lir.Branch(expression condition, expression whenTrue, expression whenFalse) - | Hir.Primitive(op, arguments) -> Lir.Operation(operation op, arguments |> List.map expression) - | Hir.Tuple items -> Lir.Tuple(items |> List.map expression) - | Hir.List items -> Lir.List(items |> List.map expression) - | Hir.Record fields -> Lir.Record(fields |> List.map (fun (name, item) -> name, expression item)) - | Hir.FieldGet(target, name) -> Lir.FieldGet(expression target, name) - | Hir.SomeValue item -> Lir.SomeValue(expression item) + state.Functions.Add + { Id = functionId; Parameter = parameter; Name = name; ParameterType = parameterType + Captures = captures; ReturnType = loweredBody.Type; Span = value.Span; Body = loweredBody } + Lir.MakeClosure(functionId, captures) + | Hir.Apply(fn, argument) -> Lir.CallClosure(expression state fn, expression state argument) + | Hir.If(condition, whenTrue, whenFalse) -> Lir.Branch(expression state condition, expression state whenTrue, expression state whenFalse) + | Hir.Primitive(op, arguments) -> Lir.Operation(operation op, arguments |> List.map (expression state)) + | Hir.Tuple items -> Lir.Tuple(items |> List.map (expression state)) + | Hir.List items -> Lir.List(items |> List.map (expression state)) + | Hir.Record fields -> Lir.Record(fields |> List.map (fun (name, item) -> name, expression state item)) + | Hir.FieldGet(target, name) -> Lir.FieldGet(expression state target, name) + | Hir.SomeValue item -> Lir.SomeValue(expression state item) | Hir.NoneValue -> Lir.NoneValue - | Hir.Cons(head, tail) -> Lir.Cons(expression head, expression tail) - | Hir.Append(left, right) -> Lir.Append(expression left, expression right) - | Hir.Let(id, name, recursive, binding, body) -> Lir.Let(symbol id, name, recursive, expression binding, expression body) + | Hir.Cons(head, tail) -> Lir.Cons(expression state head, expression state tail) + | Hir.Append(left, right) -> Lir.Append(expression state left, expression state right) + | Hir.Let(id, name, recursive, binding, body) -> Lir.Let(symbol id, name, recursive, expression state binding, expression state body) | Hir.LetRecursiveGroup(bindings, body) -> - Lir.LetRecursiveGroup(bindings |> List.map (fun (id, name, bindingType, value) -> symbol id, name, bindingType, expression value), expression body) - | Hir.Fail value -> Lir.Fail(expression value) + Lir.LetRecursiveGroup(bindings |> List.map (fun (id, name, bindingType, value) -> symbol id, name, bindingType, expression state value), expression state body) + | Hir.Fail value -> Lir.Fail(expression state value) | Hir.Match(target, cases) -> - Lir.DecisionTree(expression target, cases |> List.map (fun case -> + Lir.DecisionTree(expression state target, cases |> List.map (fun case -> ({ Pattern = pattern case.Pattern - Guard = Option.map expression case.Guard - Body = expression case.Body + Guard = Option.map (expression state) case.Guard + Body = expression state case.Body Span = case.Span }: Lir.DecisionCase))) - | Hir.Range(startValue, endValue) -> Lir.Range(expression startValue, expression endValue) - | Hir.RecordUpdate(target, fields) -> Lir.RecordUpdate(expression target, fields |> List.map (fun (name, item) -> name, expression item)) - | Hir.RecordConvert(target, targetType) -> Lir.RecordConvert(expression target, targetType) - | Hir.IndexGet(target, index) -> Lir.IndexGet(expression target, expression index) - | Hir.Interpolate parts -> Lir.Interpolate(parts |> List.map (function Hir.TextPart text -> Lir.TextPart text | Hir.ValuePart item -> Lir.ValuePart(expression item))) - | Hir.UnionCase(typeName, caseName, payload) -> Lir.UnionCase(typeName, caseName, Option.map expression payload) - | Hir.Map entries -> Lir.Map(entries |> List.map (function Hir.KeyValue(key, item) -> Lir.KeyValue(expression key, expression item) | Hir.Spread item -> Lir.Spread(expression item))) + | Hir.Range(startValue, endValue) -> Lir.Range(expression state startValue, expression state endValue) + | Hir.RecordUpdate(target, fields) -> Lir.RecordUpdate(expression state target, fields |> List.map (fun (name, item) -> name, expression state item)) + | Hir.RecordConvert(target, targetType) -> Lir.RecordConvert(expression state target, targetType) + | Hir.IndexGet(target, index) -> Lir.IndexGet(expression state target, expression state index) + | Hir.Interpolate parts -> Lir.Interpolate(parts |> List.map (function Hir.TextPart text -> Lir.TextPart text | Hir.ValuePart item -> Lir.ValuePart(expression state item))) + | Hir.UnionCase(typeName, caseName, payload) -> Lir.UnionCase(typeName, caseName, Option.map (expression state) payload) + | Hir.Map entries -> Lir.Map(entries |> List.map (function Hir.KeyValue(key, item) -> Lir.KeyValue(expression state key, expression state item) | Hir.Spread item -> Lir.Spread(expression state item))) | Hir.Intrinsic name -> Lir.Intrinsic name | Hir.HostExternal(name, arity) -> Lir.HostExternal(name, arity) | Hir.RuntimeContext -> Lir.RuntimeContext | Hir.TargetImport(modulePath, memberName) -> Lir.TargetImport(modulePath, memberName) - | Hir.ForEach(id, name, itemType, source, body) -> Lir.ForEach(symbol id, name, itemType, expression source, expression body) + | Hir.ForEach(id, name, itemType, source, body) -> Lir.ForEach(symbol id, name, itemType, expression state source, expression state body) | Hir.TypeToken tokenType -> Lir.TypeToken tokenType { Type = value.Type; Span = value.Span; Node = node } let lower (program: Hir.Program) : Lir.Program = + let state = { NextFunction = 0; Functions = ResizeArray() } + let declarations = + program.Declarations + |> List.map (fun value -> + ({ Id = symbol value.Id; Name = value.Name; Type = value.Value.Type + IsRecursive = value.IsRecursive; IsExported = value.IsExported + Span = value.Span; Value = expression state value.Value }: Lir.Declaration)) + let entry = program.Entry |> Option.map (expression state) { ImportedRecords = program.ImportedRecords |> List.map (fun value -> ({ Name = value.Name; ModulePath = value.ModulePath; MemberName = value.MemberName; IsUnion = value.IsUnion Fields = value.Fields; Span = value.Span }: Lir.ImportedRecord)) - Declarations = - program.Declarations - |> List.map (fun value -> - ({ Id = symbol value.Id; Name = value.Name; Type = value.Value.Type - IsRecursive = value.IsRecursive; IsExported = value.IsExported - Span = value.Span; Value = expression value.Value }: Lir.Declaration)) - Entry = program.Entry |> Option.map expression } + Functions = state.Functions |> Seq.sortBy (_.Id >> Lir.functionNumber) |> Seq.toList + Declarations = declarations + Entry = entry } module LirValidation = let rec private compatible left right = @@ -274,6 +304,8 @@ module LirValidation = let validate (program: Lir.Program) = let ids = program.Declarations |> List.map _.Id + let functionIds = program.Functions |> List.map _.Id + let functions = program.Functions |> List.map (fun fn -> fn.Id, fn) |> Map.ofList let invalidImports = program.ImportedRecords |> List.exists (fun record -> System.String.IsNullOrWhiteSpace(record.ModulePath) || System.String.IsNullOrWhiteSpace(record.MemberName)) || (program.ImportedRecords |> List.map _.Name |> List.distinct |> List.length) <> program.ImportedRecords.Length @@ -281,6 +313,8 @@ module LirValidation = Error [ { Code = "FSL2012"; Stage = Lowering; Message = "LIR imported record linkage must be complete and uniquely named."; Span = None } ] elif ids.Length <> (ids |> Set.ofList |> Set.count) then Error [ { Code = "FSL2001"; Stage = Lowering; Message = "LIR declaration IDs must be unique."; Span = None } ] + elif functionIds <> ([ 0 .. functionIds.Length - 1 ] |> List.map Lir.FunctionId) then + Error [ { Code = "FSL2014"; Stage = Lowering; Message = "LIR function IDs must be unique, contiguous, and ordered."; Span = None } ] else let diagnostics = ResizeArray() let report code message span = @@ -343,14 +377,16 @@ module LirValidation = match expression.Node with | Lir.Constant value when not (compatible expression.Type (literalType value)) -> report "FSL2004" "LIR constant has an inconsistent type." expression.Span | Lir.Load id when not (Set.contains id locals || Set.contains id declared) -> report "FSL2005" (sprintf "LIR reference %%%d is unresolved." (Lir.symbolNumber id)) expression.Span - | Lir.MakeClosure(parameter, _, parameterType, captures, body) -> - let expectedCaptures = LirLowering.capturedReferences (Set.singleton parameter) body |> Set.toList |> List.sortBy Lir.symbolNumber - if captures <> expectedCaptures then - report "FSL2013" "LIR closure captures do not match its ordered free-symbol set." expression.Span + | Lir.MakeClosure(functionId, captures) -> if captures |> List.exists (fun id -> not (Set.contains id locals || Set.contains id declared)) then report "FSL2005" "LIR closure contains an unresolved capture." expression.Span - checkExpression (Set.add parameter locals) body - match expression.Type with TFun(argument, result) when compatible argument parameterType && compatible result body.Type -> () | _ -> report "FSL2006" "LIR closure signature is inconsistent with its body." expression.Span + match Map.tryFind functionId functions with + | None -> report "FSL2014" "LIR closure references an unknown lifted function." expression.Span + | Some fn -> + if captures <> fn.Captures then report "FSL2013" "LIR closure captures do not match its lifted function." expression.Span + match expression.Type with + | TFun(argument, result) when compatible argument fn.ParameterType && compatible result fn.ReturnType -> () + | _ -> report "FSL2006" "LIR closure signature is inconsistent with its lifted function." expression.Span | Lir.CallClosure(fn, argument) -> check fn; check argument match fn.Type with TFun(parameter, result) when compatible parameter argument.Type && compatible result expression.Type -> () | _ -> report "FSL2006" "LIR closure call signature is inconsistent." expression.Span @@ -399,6 +435,12 @@ module LirValidation = | Lir.ForEach(id, _, itemType, source, body) -> check source; checkExpression (Set.add id locals) body; match source.Type with TList actual when compatible actual itemType && compatible expression.Type TUnit -> () | _ -> report "FSL2004" "LIR for-loop representation is inconsistent." expression.Span | Lir.TypeToken _ when not (compatible expression.Type TTypeToken) -> report "FSL2004" "LIR type token has an inconsistent type." expression.Span | _ -> () + program.Functions |> List.iter (fun fn -> + checkSpan fn.Span + let expected = LirLowering.capturedReferences (Set.singleton fn.Parameter) fn.Body |> Set.toList |> List.sortBy Lir.symbolNumber + if fn.Captures <> expected then report "FSL2013" "Lifted LIR function captures do not match its ordered free-symbol set." fn.Span + checkExpression (Set.add fn.Parameter (Set.ofList fn.Captures)) fn.Body + if not (compatible fn.ReturnType fn.Body.Type) then report "FSL2006" "Lifted LIR function return type is inconsistent." fn.Span) program.Declarations |> List.iter (fun declaration -> checkSpan declaration.Span; checkExpression Set.empty declaration.Value; if not (compatible declaration.Type declaration.Value.Type) then report "FSL2004" $"LIR declaration '{declaration.Name}' type is inconsistent." declaration.Span) program.Entry |> Option.iter (checkExpression Set.empty) if diagnostics.Count = 0 then Ok program else Error(List.ofSeq diagnostics) diff --git a/src/FScript.Interpreter.Lir/LirInterpreter.fs b/src/FScript.Interpreter.Lir/LirInterpreter.fs index 93146b3..423019a 100644 --- a/src/FScript.Interpreter.Lir/LirInterpreter.fs +++ b/src/FScript.Interpreter.Lir/LirInterpreter.fs @@ -32,6 +32,7 @@ type ExecutionResult = module LirInterpreter = let private currentOutput = AsyncLocal>() + let private currentFunctions = AsyncLocal>() let private constantValue = function | Lir.Unit -> VUnit | Lir.Int value -> VInt value | Lir.Float value -> VFloat value | Lir.Bool value -> VBool value | Lir.String value -> VString value @@ -260,9 +261,12 @@ module LirInterpreter = | Lir.Load id -> environment |> Map.tryFind id |> Option.map Ok |> Option.defaultWith (fun () -> fail "FSI3001" expression.Span (sprintf "Unknown LIR symbol %%%d." (Lir.symbolNumber id))) - | Lir.MakeClosure(id, name, _, captures, body) -> + | Lir.MakeClosure(functionId, captures) -> let captured = captures |> List.choose (fun symbol -> Map.tryFind symbol environment |> Option.map (fun value -> symbol, value)) |> Map.ofList - Ok(VClosure(id, name, body, ref captured)) + currentFunctions.Value + |> Map.tryFind functionId + |> Option.map (fun fn -> Ok(VClosure(fn.Parameter, fn.Name, fn.Body, ref captured))) + |> Option.defaultWith (fun () -> fail "FSI3024" expression.Span $"Unknown lifted LIR function {Lir.functionNumber functionId}.") | Lir.CallClosure(fn, argument) -> match evaluate environment fn, evaluate environment argument with | Ok fnValue, Ok value -> applyValue expression.Span fnValue value @@ -457,6 +461,7 @@ module LirInterpreter = let execute (context: InterpreterContext) (program: Lir.Program) = let output = ResizeArray() currentOutput.Value <- output + currentFunctions.Value <- program.Functions |> List.map (fun fn -> fn.Id, fn) |> Map.ofList let runtimeContext = VRecord(Map.ofList [ "ScriptName", VOption(Option.map VString context.ScriptName) diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index b87f794..0662109 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -37,19 +37,19 @@ type HirTests() = let branch: Lir.Expression = { Type = TInt; Span = span Node = Lir.Branch(constant TInt (Lir.Int 1L), constant TInt (Lir.Int 2L), constant TString (Lir.String "no")) } - match LirValidation.validate { ImportedRecords = []; Declarations = []; Entry = Some branch } with + match LirValidation.validate { ImportedRecords = []; Functions = []; Declarations = []; Entry = Some branch } with | Ok _ -> Assert.Fail("Expected invalid LIR diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSL2007")) [] member _.``LIR validation rejects unknown primitives`` () = let intrinsic = ({ Type = TInt; Span = span; Node = Lir.Intrinsic "Unknown.value" }: Lir.Expression) - match LirValidation.validate { ImportedRecords = []; Declarations = []; Entry = Some intrinsic } with + match LirValidation.validate { ImportedRecords = []; Functions = []; Declarations = []; Entry = Some intrinsic } with | Ok _ -> Assert.Fail("Expected unavailable primitive diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSL2009")) [] member _.``Initial LIR optimization pass is identity`` () = let expression = ({ Type = TInt; Span = span; Node = Lir.Constant(Lir.Int 1L) }: Lir.Expression) - let program: Lir.Program = { ImportedRecords = []; Declarations = []; Entry = Some expression } + let program: Lir.Program = { ImportedRecords = []; Functions = []; Declarations = []; Entry = Some expression } Assert.That(LirOptimization.optimize program = program, Is.True) diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index b8acbe3..b33d382 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -113,7 +113,10 @@ type IrArtifactTests() = [] member _.``LIR closures serialize deterministic explicit captures`` () = let source = "let outer x = (let inner y = x + y\n inner\n)\nouter 40 2" - let bytes = (compile source).Lir |> IrArtifact.serialize + let lir = (compile source).Lir + Assert.That((lir.Functions |> List.map (fun fn -> Lir.functionNumber fn.Id)) = [ 0; 1 ], Is.True) + Assert.That(lir.Functions |> List.exists (fun fn -> not fn.Captures.IsEmpty), Is.True) + let bytes = lir |> IrArtifact.serialize let json = Encoding.UTF8.GetString(bytes) Assert.That(json, Does.Match("\\\"captures\\\": \\[\\s+[0-9]+")) match IrArtifact.deserialize bytes with From 7b95347e907b01ad576ac2c8f03fb101184b59a5 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 19:14:27 +0200 Subject: [PATCH 040/133] feat(cli): harden compiler output commands --- CHANGELOG.md | 2 + docs/README.md | 1 + docs/guides/compiler-cli.md | 14 +++++++ src/FScript/Program.fs | 65 ++++++++++++++++------------- tests/FScript.Cli.Tests/CliTests.fs | 29 ++++++++++++- 5 files changed, 80 insertions(+), 31 deletions(-) create mode 100644 docs/guides/compiler-cli.md diff --git a/CHANGELOG.md b/CHANGELOG.md index a041bb2..0129704 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Documented the compiler CLI and made IR and Go output replacement deterministic with early target validation. + - Lifted LIR lambda bodies into a deterministic validated function table consumed by artifacts, interpretation, and Go generation. - Added deterministic explicit closure captures to LIR serialization, validation, and interpretation. diff --git a/docs/README.md b/docs/README.md index e148629..2f0382c 100644 --- a/docs/README.md +++ b/docs/README.md @@ -10,6 +10,7 @@ This folder is organized into three layers: - Tutorial: [`guides/getting-started-tutorial.md`](./guides/getting-started-tutorial.md) - REPL guide: [`guides/repl.md`](./guides/repl.md) +- Compiler CLI: [`guides/compiler-cli.md`](./guides/compiler-cli.md) - FScript vs F# / OCaml: [`guides/fsharp-ocaml-differences.md`](./guides/fsharp-ocaml-differences.md) - Language rationale: [`guides/language-choices.md`](./guides/language-choices.md) diff --git a/docs/guides/compiler-cli.md b/docs/guides/compiler-cli.md new file mode 100644 index 0000000..ccb3739 --- /dev/null +++ b/docs/guides/compiler-cli.md @@ -0,0 +1,14 @@ +# Compiler CLI + +The compiler emits either a versioned FScript LIR artifact or a complete Go source project. + +```text +fscript compile --target ir --output +fscript compile --target go --output +``` + +The IR form writes `/.fsir`. The Go form writes a project rooted at the requested directory, including `go.mod`, `main.go`, `internal/fscriptprogram/program.go`, and the embedded runtime under `internal/fsruntime/`. + +Both forms create missing parent directories and deterministically replace files owned by the selected output on a repeated invocation. An unknown target is rejected before the output directory is created. Filesystem failures are reported as `FSA5101` diagnostics. + +Generated projects declare `go 1.25`, use no Go generics, and are already formatted for an empty `gofmt -d` result. Generation itself does not invoke Go; building or running the resulting project remains the caller's responsibility. diff --git a/src/FScript/Program.fs b/src/FScript/Program.fs index 928e503..6a6145c 100644 --- a/src/FScript/Program.fs +++ b/src/FScript/Program.fs @@ -26,36 +26,41 @@ let runCompileCommand (argv: string array) = |> Option.bind (fun index -> if index + 1 < argv.Length then Some argv.[index + 1] else None) match tryOption "--target", tryOption "--output" with | Some target, Some output when File.Exists scriptPath -> - try - let outputDirectory = Path.GetFullPath(output) - let sourceRoot = Path.GetDirectoryName(scriptPath) |> Option.ofObj |> Option.defaultValue (Directory.GetCurrentDirectory()) - let program = FScript.parseFileWithIncludes sourceRoot scriptPath - match FScript.Compiler.Compiler.compile { Program = program; Externs = [] } with - | Error diagnostics -> - diagnostics |> List.iter (fun diagnostic -> Console.Error.WriteLine($"{diagnostic.Code}: {diagnostic.Message}")) - 4 - | Ok compiled -> - Directory.CreateDirectory(outputDirectory) |> ignore - match target.ToLowerInvariant() with - | "ir" -> - let baseName = - match Path.GetFileNameWithoutExtension(scriptPath) with - | null | "" -> "program" - | value -> value - let name = baseName + ".fsir" - File.WriteAllBytes(Path.Combine(outputDirectory, name), FScript.Compiler.IrArtifact.serialize compiled.Lir) - 0 - | "go" -> - match FScript.Compiler.GoBackend.generate { ModuleName = "fscript.generated/program" } compiled.Lir with - | Error diagnostics -> - diagnostics |> List.iter (fun diagnostic -> Console.Error.WriteLine($"{diagnostic.Code}: {diagnostic.Message}")) - 4 - | Ok files -> files |> List.iter (writeGeneratedFile outputDirectory); 0 - | _ -> usage () - with - | ParseException error -> Console.Error.WriteLine($"Parse error: {error.Message}"); 1 - | TypeException error -> Console.Error.WriteLine($"Type error: {error.Message}"); 2 - | ex -> Console.Error.WriteLine(ex.Message); 4 + let normalizedTarget = target.ToLowerInvariant() + if normalizedTarget <> "ir" && normalizedTarget <> "go" then usage () + else + try + let outputDirectory = Path.GetFullPath(output) + let sourceRoot = Path.GetDirectoryName(scriptPath) |> Option.ofObj |> Option.defaultValue (Directory.GetCurrentDirectory()) + let program = FScript.parseFileWithIncludes sourceRoot scriptPath + match FScript.Compiler.Compiler.compile { Program = program; Externs = [] } with + | Error diagnostics -> + diagnostics |> List.iter (fun diagnostic -> Console.Error.WriteLine($"{diagnostic.Code}: {diagnostic.Message}")) + 4 + | Ok compiled -> + Directory.CreateDirectory(outputDirectory) |> ignore + match normalizedTarget with + | "ir" -> + let baseName = + match Path.GetFileNameWithoutExtension(scriptPath) with + | null | "" -> "program" + | value -> value + let name = baseName + ".fsir" + File.WriteAllBytes(Path.Combine(outputDirectory, name), FScript.Compiler.IrArtifact.serialize compiled.Lir) + 0 + | "go" -> + match FScript.Compiler.GoBackend.generate { ModuleName = "fscript.generated/program" } compiled.Lir with + | Error diagnostics -> + diagnostics |> List.iter (fun diagnostic -> Console.Error.WriteLine($"{diagnostic.Code}: {diagnostic.Message}")) + 4 + | Ok files -> files |> List.iter (writeGeneratedFile outputDirectory); 0 + | _ -> usage () + with + | ParseException error -> Console.Error.WriteLine($"Parse error: {error.Message}"); 1 + | TypeException error -> Console.Error.WriteLine($"Type error: {error.Message}"); 2 + | :? UnauthorizedAccessException as ex -> Console.Error.WriteLine($"FSA5101: Cannot write compiler output: {ex.Message}"); 4 + | :? IOException as ex -> Console.Error.WriteLine($"FSA5101: Cannot write compiler output: {ex.Message}"); 4 + | ex -> Console.Error.WriteLine(ex.Message); 4 | Some _, Some _ -> Console.Error.WriteLine($"File not found: {scriptPath}"); 1 | _ -> usage () diff --git a/tests/FScript.Cli.Tests/CliTests.fs b/tests/FScript.Cli.Tests/CliTests.fs index ac7fdda..1dc4efe 100644 --- a/tests/FScript.Cli.Tests/CliTests.fs +++ b/tests/FScript.Cli.Tests/CliTests.fs @@ -122,7 +122,12 @@ type CliTests() = Assert.That(code, Is.EqualTo(0), stderr) let artifact = Path.Combine(output, "program.fsir") Assert.That(File.Exists artifact, Is.True) + let expected = File.ReadAllBytes(artifact) Assert.That(File.ReadAllText(artifact), Does.Contain("\"format\": \"fscript-lir\"")) + File.WriteAllText(artifact, "stale") + let secondCode, _, secondError = runCli repoRoot repoRoot [ "compile"; script; "--target"; "ir"; "--output"; output ] None + Assert.That(secondCode, Is.EqualTo(0), secondError) + Assert.That(File.ReadAllBytes(artifact) = expected, Is.True, "IR compilation must deterministically replace an existing artifact") finally if Directory.Exists tempRoot then Directory.Delete(tempRoot, true) @@ -138,7 +143,29 @@ type CliTests() = let code, _, stderr = runCli repoRoot repoRoot [ "compile"; script; "--target"; "go"; "--output"; output ] None Assert.That(code, Is.EqualTo(0), stderr) Assert.That(File.Exists(Path.Combine(output, "go.mod")), Is.True) - Assert.That(File.Exists(Path.Combine(output, "internal", "fscriptprogram", "program.go")), Is.True) + let programFile = Path.Combine(output, "internal", "fscriptprogram", "program.go") + Assert.That(File.Exists(programFile), Is.True) + let expected = File.ReadAllText(programFile) + File.WriteAllText(programFile, "stale") + let secondCode, _, secondError = runCli repoRoot repoRoot [ "compile"; script; "--target"; "go"; "--output"; output ] None + Assert.That(secondCode, Is.EqualTo(0), secondError) + Assert.That(File.ReadAllText(programFile), Is.EqualTo(expected), "Go compilation must deterministically replace generated files") + finally + if Directory.Exists tempRoot then Directory.Delete(tempRoot, true) + + [] + member _.``Invalid compile target does not create output`` () = + let repoRoot = findRepoRoot () + let tempRoot = Path.Combine(Path.GetTempPath(), $"fscript-compile-invalid-{Guid.NewGuid():N}") + let script = Path.Combine(tempRoot, "program.fss") + let output = Path.Combine(tempRoot, "out") + try + Directory.CreateDirectory(tempRoot) |> ignore + File.WriteAllText(script, "1") + let code, _, stderr = runCli repoRoot repoRoot [ "compile"; script; "--target"; "unknown"; "--output"; output ] None + Assert.That(code, Is.EqualTo(1)) + Assert.That(stderr, Does.Contain("Usage: fscript compile")) + Assert.That(Directory.Exists(output), Is.False) finally if Directory.Exists tempRoot then Directory.Delete(tempRoot, true) From 0da1967064dd18d714814e38be1276d5ff713fc9 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 19:16:15 +0200 Subject: [PATCH 041/133] feat(go): reject reachable host externs --- CHANGELOG.md | 2 ++ docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Backend.Go/GoBackend.fs | 17 +++++++++++++---- .../FScript.Backend.Go.Tests/GoBackendTests.fs | 16 ++++++++++++++++ 4 files changed, 32 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0129704..8597a0e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added reachability-sensitive Go diagnostics for unsupported host externs without blocking unused descriptors. + - Documented the compiler CLI and made IR and Go output replacement deterministic with early target validation. - Lifted LIR lambda bodies into a deterministic validated function table consumed by artifacts, interpretation, and Go generation. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 337c061..c0cc4a5 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -45,6 +45,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - HIR and LIR are accepted only when artifact-local references resolve, source spans are valid, declaration and expression types agree, calls and control-flow nodes have consistent signatures, aggregate representations match their semantic types, and referenced primitives/imports are available and complete. - Task signatures and host linkage remain representable in compiled artifacts. An unused task function handle is inert; fully applying `Task.spawn` or `Task.await` in LIR reports `FSI3021`, while a Go program whose reachable declaration graph requires a task reports `FSB4004`. - Compiler extern input is declarative: it carries a name, type scheme, arity, and linkage classification, never an executable `.NET` delegate. Host execution values remain confined to the legacy/runtime API. +- Generated Go rejects a reachable arbitrary host extern with `FSB4006`; an unused host-extern descriptor remains inert and does not block generation. - `Console.writeLine` evaluates its string argument once, appends one stdout line, and returns unit. LIR reports captured lines in order through `ExecutionResult.Stdout`; generated Go emits them in the same order before its final formatted result. - Generated Go source is deterministic and already canonical according to `gofmt`; generation itself has no installed-Go dependency. Local integration tests skip only when Go is unavailable, while CI installs Go 1.25 and treats its absence as a failure. - Tuples and lists remain observably distinct in generated Go: tuples format with parentheses, lists with brackets, tuple patterns require tuple values, and map pair inputs consume tuple values. diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index d090b2c..b32890e 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -603,6 +603,14 @@ func Environment() map[string]any { program.Declarations |> List.tryPick (fun declaration -> if Set.contains declaration.Id reachable && containsTask declaration.Type then Some declaration.Span else None) |> Option.orElseWith (fun () -> program.Entry |> Option.bind (fun entry -> if containsTask entry.Type then Some entry.Span else None)) + let reachableHostExternal = + program.Declarations + |> List.tryPick (fun declaration -> + if Set.contains declaration.Id reachable then + match declaration.Value.Node with + | Lir.HostExternal(name, _) when not (containsTask declaration.Type) -> Some(name, declaration.Span) + | _ -> None + else None) let invalidImportedRecord = program.ImportedRecords |> List.tryFind (fun record -> @@ -611,7 +619,8 @@ func Environment() map[string]any { || (record.Fields |> List.exists (fun (name, fieldType) -> let unsupportedType = match fieldType with TUnit | TInt | TFloat | TBool | TString -> false | _ -> true System.String.IsNullOrEmpty(name) || not (System.Char.IsUpper(name.[0])) || containsTask fieldType || unsupportedType))) - match invalidImportedRecord, taskSpan with - | Some record, _ -> Error [ { Code = "FSB4005"; Stage = GoGeneration; Message = $"Imported Go record '{record.Name}' must have exported, exactly named primitive fields and cannot be a union."; Span = Some record.Span } ] - | None, Some span -> Error [ { Code = "FSB4004"; Stage = GoGeneration; Message = "Task values are not supported by the Go backend."; Span = Some span } ] - | None, None -> generateCore options program + match invalidImportedRecord, taskSpan, reachableHostExternal with + | Some record, _, _ -> Error [ { Code = "FSB4005"; Stage = GoGeneration; Message = $"Imported Go record '{record.Name}' must have exported, exactly named primitive fields and cannot be a union."; Span = Some record.Span } ] + | None, Some span, _ -> Error [ { Code = "FSB4004"; Stage = GoGeneration; Message = "Task values are not supported by the Go backend."; Span = Some span } ] + | None, None, Some(name, span) -> Error [ { Code = "FSB4006"; Stage = GoGeneration; Message = $"Host external '{name}' is not supported by the Go backend."; Span = Some span } ] + | None, None, None -> generateCore options program diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index 91e8b74..4518293 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -32,6 +32,22 @@ type GoBackendTests() = | Ok _ -> Assert.Fail("Expected unsupported task diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSB4004")) + [] + member _.``Go backend rejects only reachable host externs`` () = + let external = + { Name = "Host.value"; Scheme = Forall([], TFun(TInt, TInt)); Arity = 1 + Kind = UnsupportedHostExtern } + let compile source = + match Compiler.compile { Program = Parser.parseProgram source; Externs = [ external ] } with + | Ok value -> value.Lir + | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + match GoBackend.generate { ModuleName = "fscript.generated/externs" } (compile "Host.value 1") with + | Ok _ -> Assert.Fail("Expected unsupported host-extern diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSB4006")) + match GoBackend.generate { ModuleName = "fscript.generated/externs" } (compile "1") with + | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) + | Ok _ -> () + [] member _.``Go backend emits imported record boundary adapters`` () = let source = "[] type Person = { Name: string; Age: int }\n[] let birthday : Person -> Person\nbirthday { Name = \"Ada\"; Age = 42 }" From fe8eb5fb50245d5a3a14ce5b04c3c96704e22ef2 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 19:22:09 +0200 Subject: [PATCH 042/133] feat(compiler): preserve structured export metadata --- CHANGELOG.md | 2 ++ docs/architecture/typed-ir-and-backends.md | 2 +- src/FScript.Compiler/Hir.fs | 8 +++++++- src/FScript.Compiler/HirLowering.fs | 16 +++++++++++----- src/FScript.Compiler/HirValidation.fs | 2 ++ src/FScript.Compiler/IrArtifact.fs | 12 ++++++++++-- src/FScript.Compiler/Lir.fs | 9 +++++++++ src/FScript.Language/TypeInfer.fs | 13 ++++++++++++- tests/FScript.Compiler.Tests/HirTests.fs | 2 +- tests/FScript.Compiler.Tests/IrArtifactTests.fs | 11 +++++++++++ .../ResolvedInferenceTests.fs | 9 +++++++++ 11 files changed, 75 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8597a0e..297f746 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Preserved structured export names and source spans through resolved programs, HIR, LIR, and artifacts. + - Added reachability-sensitive Go diagnostics for unsupported host externs without blocking unused descriptors. - Documented the compiler CLI and made IR and Go output replacement deterministic with early target validation. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 897d848..a78c499 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -20,7 +20,7 @@ Validated LIR consumers are separate leaf assemblies. `FScript.Interpreter.Lir` ## IR boundaries -HIR is language-oriented and contains resolved artifact-local symbol identities, complete semantic types and schemes, source locations, declaration attributes, and explicit nodes for named/structural record conversions. Conversion evidence is produced by inference, not rediscovered from backend representations. HIR contains no Go types. +HIR is language-oriented and contains resolved artifact-local symbol identities, complete semantic types and schemes, source locations, structured export metadata, and explicit nodes for named/structural record conversions. Export metadata retains the exported source name and span through LIR and artifacts while the legacy Boolean remains available to compatibility consumers. Conversion evidence is produced by inference, not rediscovered from backend representations. HIR contains no Go types. LIR is executable but not a Go AST. Lambda bodies are lifted into a deterministic program-level function table. Closure creation references a function ID and carries its canonical ordered set of captured symbol IDs, alongside resolved primitive operations, aggregate construction/projection, and structured control flow. Validation recomputes each lifted function's free-symbol set and rejects unknown function IDs or forged, missing, duplicate, or unordered captures. Pattern matching becomes deterministic tests and projections before reaching a backend. Semantic types remain alongside selected runtime representations for validation and diagnostics. diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs index f3ca83c..b12dd61 100644 --- a/src/FScript.Compiler/Hir.fs +++ b/src/FScript.Compiler/Hir.fs @@ -107,9 +107,14 @@ module Hir = Scheme: Scheme IsRecursive: bool IsExported: bool + Export: ExportMetadata option Span: Span Value: Expression } + and ExportMetadata = + { Name: string + Span: Span } + type ImportedRecord = { Name: string ModulePath: string @@ -208,7 +213,8 @@ module Hir = | TypeToken tokenType -> lines.Add(typed $"typeof {formatType tokenType}") for declaration in program.Declarations do - lines.Add(sprintf "decl %%%d %s: %s" (symbolNumber declaration.Id) declaration.Name (formatType (let (Forall(_, value)) = declaration.Scheme in value))) + let export = declaration.Export |> Option.map (fun metadata -> $" export={metadata.Name}") |> Option.defaultValue "" + lines.Add(sprintf "decl %%%d %s%s: %s" (symbolNumber declaration.Id) declaration.Name export (formatType (let (Forall(_, value)) = declaration.Scheme in value))) expression 2 declaration.Value match program.Entry with | Some entry -> diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index ea230f4..d1e430f 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -327,6 +327,7 @@ module HirLowering = |> collectResults |> Result.map (Hir.Map >> make) let lower (externs: CompilerExtern list) (resolved: TypeInfer.ResolvedProgram) : Result = + let exportMetadata name span enabled = if enabled then Some({ Name = name; Span = span }: Hir.ExportMetadata) else None let expressionTypes = let rec addExpression state (item: TypeInfer.ResolvedExpression) = item.Children |> List.fold addExpression (Map.add item.Source item.Type state) @@ -393,7 +394,7 @@ module HirLowering = BuiltinSignatures.builtinSchemes |> Map.toList |> List.sortBy fst |> List.map (fun (name, scheme) -> let (Forall(_, valueType)) = scheme let span = Span.mk (Span.pos 0 0) (Span.pos 0 0) - ({ Id = builtins.[name]; Name = name; Scheme = scheme; IsRecursive = false; IsExported = false; Span = span; Value = { Type = valueType; Span = span; Node = Hir.Intrinsic name } }: Hir.Declaration)) + ({ Id = builtins.[name]; Name = name; Scheme = scheme; IsRecursive = false; IsExported = false; Export = None; Span = span; Value = { Type = valueType; Span = span; Node = Hir.Intrinsic name } }: Hir.Declaration)) let externalDeclarations = externs |> List.sortBy _.Name |> List.map (fun external -> let (Forall(_, valueType)) = external.Scheme @@ -403,7 +404,7 @@ module HirLowering = | RuntimeContextIntrinsic -> Hir.RuntimeContext | PortableRuntimeIntrinsic -> Hir.Intrinsic external.Name | UnsupportedHostExtern -> Hir.HostExternal(external.Name, external.Arity) - ({ Id = externSymbols.[external.Name]; Name = external.Name; Scheme = external.Scheme; IsRecursive = false; IsExported = false; Span = span; Value = { Type = valueType; Span = span; Node = node } }: Hir.Declaration)) + ({ Id = externSymbols.[external.Name]; Name = external.Name; Scheme = external.Scheme; IsRecursive = false; IsExported = false; Export = None; Span = span; Value = { Type = valueType; Span = span; Node = node } }: Hir.Declaration)) builtinDeclarations @ externalDeclarations @ (constructorSymbols |> List.map (fun (typeName, caseName, id) -> let definition, unionType = unionDefinitions |> List.find (fun (definition, _) -> definition.Name = typeName) let payloadType = definition.Cases |> List.find (fst >> (=) caseName) |> snd |> Option.map typeFromRef @@ -414,7 +415,7 @@ module HirLowering = let parameter = newSymbol state let body = ({ Type = unionType; Span = definition.Span; Node = Hir.UnionCase(typeName, caseName, Some({ Type = itemType; Span = definition.Span; Node = Hir.Reference parameter })) }: Hir.Expression) ({ Type = TFun(itemType, unionType); Span = definition.Span; Node = Hir.Lambda(parameter, "value", itemType, body) }: Hir.Expression) - ({ Id = id; Name = $"{typeName}.{caseName}"; Scheme = Types.generalize Map.empty value.Type; IsRecursive = false; IsExported = false; Span = definition.Span; Value = value }: Hir.Declaration))) + ({ Id = id; Name = $"{typeName}.{caseName}"; Scheme = Types.generalize Map.empty value.Type; IsRecursive = false; IsExported = false; Export = None; Span = definition.Span; Value = value }: Hir.Declaration))) let mutable entry: Hir.Expression option = None let mutable errors = [] for statement in resolved.TypedProgram do @@ -428,6 +429,7 @@ module HirLowering = Scheme = Types.generalize Map.empty inferred; IsRecursive = isRecursive; IsExported = isExported; + Export = (exportMetadata name span isExported); Span = span; Value = value }: Hir.Declaration) ] | Error diagnostics -> errors <- errors @ diagnostics @@ -442,6 +444,7 @@ module HirLowering = Scheme = Types.generalize Map.empty expression.Type IsRecursive = false IsExported = false + Export = None Span = value.Span Value = value }: Hir.Declaration) ] entry <- Some { Type = expression.Type; Span = value.Span; Node = Hir.Reference id } @@ -457,6 +460,7 @@ module HirLowering = Scheme = Types.generalize Map.empty bindingType IsRecursive = false IsExported = false + Export = None Span = span Value = value }: Hir.Declaration) ] let target = ({ Type = bindingType; Span = span; Node = Hir.Reference temporaryId }: Hir.Expression) @@ -464,7 +468,7 @@ module HirLowering = let check = ({ Type = TUnit; Span = span; Node = Hir.Match(target, [ { Pattern = loweredPattern; Guard = None; Body = unitBody; Span = span } ]) }: Hir.Expression) let checkId = newSymbol state declarations <- declarations @ - [ ({ Id = checkId; Name = $"$patternCheck{Hir.symbolNumber checkId}"; Scheme = Forall([], TUnit); IsRecursive = false; IsExported = false; Span = span; Value = check }: Hir.Declaration) ] + [ ({ Id = checkId; Name = $"$patternCheck{Hir.symbolNumber checkId}"; Scheme = Forall([], TUnit); IsRecursive = false; IsExported = false; Export = None; Span = span; Value = check }: Hir.Declaration) ] for name, inferred in bindings |> Map.toList do match patternBindings |> Map.tryFind name with | None -> errors <- errors @ [ Diagnostic.hir "FSH1006" $"Pattern binding '{name}' was not resolved." (Some span) ] @@ -472,7 +476,7 @@ module HirLowering = let body = ({ Type = inferred; Span = span; Node = Hir.Reference patternId }: Hir.Expression) let projection = ({ Type = inferred; Span = span; Node = Hir.Match(target, [ { Pattern = loweredPattern; Guard = None; Body = body; Span = span } ]) }: Hir.Expression) declarations <- declarations @ - [ ({ Id = Map.find name topLevel; Name = name; Scheme = Types.generalize Map.empty inferred; IsRecursive = false; IsExported = isExported; Span = span; Value = projection }: Hir.Declaration) ] + [ ({ Id = Map.find name topLevel; Name = name; Scheme = Types.generalize Map.empty inferred; IsRecursive = false; IsExported = isExported; Export = (exportMetadata name span isExported); Span = span; Value = projection }: Hir.Declaration) ] | Error a, Error b -> errors <- errors @ a @ b | Error diagnostics, _ | _, Error diagnostics -> errors <- errors @ diagnostics | TypeInfer.TSLetRecGroup(bindings, isExported, groupSpan) -> @@ -485,6 +489,7 @@ module HirLowering = Scheme = Types.generalize Map.empty inferred IsRecursive = true IsExported = isExported + Export = (exportMetadata name span isExported) Span = span Value = value }: Hir.Declaration) ] | Error diagnostics -> errors <- errors @ diagnostics @@ -495,6 +500,7 @@ module HirLowering = Scheme = Forall([], importedType) IsRecursive = false IsExported = false + Export = None Span = span Value = { Type = importedType; Span = span; Node = Hir.TargetImport(import.ModulePath, import.MemberName) } }: Hir.Declaration) ] | TypeInfer.TSType _ -> () diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index fd5523d..82d0ba8 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -30,6 +30,8 @@ module HirValidation = Error [ Diagnostic.hir "FSH1105" "HIR imported record linkage must be complete and uniquely named." None ] elif not duplicateIds.IsEmpty then Error [ Diagnostic.hir "FSH1101" "HIR declaration IDs must be unique." None ] + elif program.Declarations |> List.exists (fun declaration -> declaration.IsExported <> declaration.Export.IsSome || (declaration.Export |> Option.exists (fun export -> export.Name <> declaration.Name || not (validSpan export.Span)))) then + Error [ Diagnostic.hir "FSH1106" "HIR export metadata is inconsistent." None ] elif program.Declarations |> List.exists (fun declaration -> not (validSpan declaration.Span && validSpan declaration.Value.Span)) then Error [ Diagnostic.hir "FSH1103" "HIR contains an invalid source span." None ] elif program.Declarations |> List.exists (fun declaration -> let (Forall(_, declaredType)) = declaration.Scheme in not (compatible declaredType declaration.Value.Type)) then diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index 5c441af..eb82d46 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -277,7 +277,11 @@ module IrArtifact = for declaration in program.Declarations do writer.WriteStartObject(); writer.WriteNumber("id", Lir.symbolNumber declaration.Id); writer.WriteString("name", declaration.Name) writer.WritePropertyName("type"); writeType writer declaration.Type; writer.WriteBoolean("recursive", declaration.IsRecursive) - writer.WriteBoolean("exported", declaration.IsExported); writer.WritePropertyName("span"); writeSpan writer declaration.Span + writer.WriteBoolean("exported", declaration.IsExported) + match declaration.Export with + | Some export -> writer.WriteStartObject("export"); writer.WriteString("name", export.Name); writer.WritePropertyName("span"); writeSpan writer export.Span; writer.WriteEndObject() + | None -> writer.WriteNull("export") + writer.WritePropertyName("span"); writeSpan writer declaration.Span writer.WritePropertyName("value"); writeExpression writer declaration.Value; writer.WriteEndObject() writer.WriteEndArray() match program.Entry with Some entry -> writer.WritePropertyName("entry"); writeExpression writer entry | None -> writer.WriteNull("entry") @@ -449,8 +453,12 @@ module IrArtifact = Fields = (property "fields" item).EnumerateArray() |> Seq.map (fun field -> stringProperty "name" field, readType (property "type" field)) |> Seq.toList Span = readSpan (property "span" item) }: Lir.ImportedRecord)) |> Seq.toList let declarations = (property "declarations" programElement).EnumerateArray() |> Seq.map (fun (item: JsonElement) -> + let exportElement = property "export" item + let export = + if exportElement.ValueKind = JsonValueKind.Null then None + else Some({ Name = stringProperty "name" exportElement; Span = readSpan (property "span" exportElement) }: Lir.ExportMetadata) ({ Id = Lir.SymbolId((property "id" item).GetInt32()); Name = stringProperty "name" item; Type = readType (property "type" item) - IsRecursive = (property "recursive" item).GetBoolean(); IsExported = (property "exported" item).GetBoolean() + IsRecursive = (property "recursive" item).GetBoolean(); IsExported = (property "exported" item).GetBoolean(); Export = export Span = readSpan (property "span" item); Value = readExpression (property "value" item) }: Lir.Declaration)) |> Seq.toList let functions = (property "functions" programElement).EnumerateArray() |> Seq.map (fun (item: JsonElement) -> ({ Id = Lir.FunctionId((property "id" item).GetInt32()) diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index d6b975a..30059c3 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -96,9 +96,14 @@ module Lir = Type: Type IsRecursive: bool IsExported: bool + Export: ExportMetadata option Span: Span Value: Expression } + and ExportMetadata = + { Name: string + Span: Span } + type Function = { Id: FunctionId Parameter: SymbolId @@ -273,6 +278,7 @@ module LirLowering = |> List.map (fun value -> ({ Id = symbol value.Id; Name = value.Name; Type = value.Value.Type IsRecursive = value.IsRecursive; IsExported = value.IsExported + Export = value.Export |> Option.map (fun export -> { Name = export.Name; Span = export.Span }) Span = value.Span; Value = expression state value.Value }: Lir.Declaration)) let entry = program.Entry |> Option.map (expression state) { ImportedRecords = @@ -325,6 +331,9 @@ module LirValidation = || span.End.Line > span.Start.Line || (span.End.Line = span.Start.Line && span.End.Column >= span.Start.Column)) let checkSpan span = if not (validSpan span) then report "FSL2002" "LIR source span is invalid." span + program.Declarations |> List.iter (fun declaration -> + if declaration.IsExported <> declaration.Export.IsSome || (declaration.Export |> Option.exists (fun export -> export.Name <> declaration.Name || not (validSpan export.Span))) then + report "FSL2015" "LIR export metadata is inconsistent." declaration.Span) let rec checkPattern expected (pattern: Lir.Pattern) = checkSpan pattern.Span if not (compatible expected pattern.Type) then report "FSL2003" "LIR pattern type does not match its scrutinee." pattern.Span diff --git a/src/FScript.Language/TypeInfer.fs b/src/FScript.Language/TypeInfer.fs index 964441e..211d0f6 100644 --- a/src/FScript.Language/TypeInfer.fs +++ b/src/FScript.Language/TypeInfer.fs @@ -225,8 +225,13 @@ module TypeInfer = type ResolvedStatement = { Source: TypedStmt + Export: ResolvedExport option Expressions: ResolvedExpression list } + and ResolvedExport = + { Name: string + Span: Span } + type ResolvedProgram = { TypedProgram: TypedProgram ResolvedStatements: ResolvedStatement list @@ -1690,7 +1695,13 @@ module TypeInfer = | TSLetRecGroup(bindings, _, _) -> bindings |> List.map (fun (_, expression, _, _) -> expression) | TSExpr expression -> [ expression.Expr ] | TSBackendImport _ | TSType _ -> [] - { Source = statement; Expressions = List.map resolveExpression roots }) + let export = + match statement with + | TSLet(name, _, _, _, true, span) -> Some { Name = name; Span = span } + | TSLetPattern(_, _, _, true, span) -> Some { Name = ""; Span = span } + | TSLetRecGroup(bindings, true, span) -> Some { Name = bindings |> List.map (fun (name, _, _, _) -> name) |> String.concat ","; Span = span } + | _ -> None + { Source = statement; Export = export; Expressions = List.map resolveExpression roots }) { TypedProgram = typed ResolvedStatements = resolvedStatements ExpressionTypes = expressions diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index 0662109..031c365 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -26,7 +26,7 @@ type HirTests() = let value: Hir.Expression = { Type = TInt; Span = span; Node = Hir.Constant(Hir.Int 1L) } let declaration: Hir.Declaration = { Id = Hir.SymbolId 0; Name = "value"; Scheme = Forall([], TString) - IsRecursive = false; IsExported = false; Span = span; Value = value } + IsRecursive = false; IsExported = false; Export = None; Span = span; Value = value } match HirValidation.validate { ImportedRecords = []; Declarations = [ declaration ]; Entry = None } with | Ok _ -> Assert.Fail("Expected invalid HIR diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSH1104")) diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index b33d382..2743da9 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -132,3 +132,14 @@ type IrArtifactTests() = match IrArtifact.deserialize invalid with | Ok _ -> Assert.Fail("Expected invalid closure-capture diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSL2013")) + + [] + member _.``IR artifact preserves structured export metadata`` () = + let lir = (compile "[] let answer = 42").Lir + let exported = lir.Declarations |> List.find _.IsExported + Assert.That(exported.Export |> Option.map _.Name, Is.EqualTo(Some "answer")) + match lir |> IrArtifact.serialize |> IrArtifact.deserialize with + | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) + | Ok loaded -> + let roundTripped = loaded.Declarations |> List.find _.IsExported + Assert.That(roundTripped.Export, Is.EqualTo(exported.Export)) diff --git a/tests/FScript.Language.Tests/ResolvedInferenceTests.fs b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs index 9babaf9..f8ed7eb 100644 --- a/tests/FScript.Language.Tests/ResolvedInferenceTests.fs +++ b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs @@ -74,3 +74,12 @@ type ResolvedInferenceTests() = Assert.That(evidence.TargetName, Is.EqualTo(Some "Person")) match evidence.SourceType with TRecord fields -> Assert.That(fields.Count, Is.EqualTo(2)) | _ -> Assert.Fail("Expected structural record source") | None -> Assert.Fail("Expected structural-to-named record coercion evidence") + + [] + member _.``Resolved statements preserve structured export metadata`` () = + let resolved = Helpers.parse "[] let answer = 42" |> TypeInfer.inferResolvedProgram + match resolved.ResolvedStatements |> List.tryPick _.Export with + | Some export -> + Assert.That(export.Name, Is.EqualTo("answer")) + Assert.That(export.Span.Start.Line, Is.EqualTo(1)) + | None -> Assert.Fail("Expected resolved export metadata") From 6531d82919422bbbe38d03bddea624050771ba29 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 19:26:10 +0200 Subject: [PATCH 043/133] test(compiler): snapshot canonical typed IR --- CHANGELOG.md | 2 + docs/architecture/typed-ir-and-backends.md | 2 + src/FScript.Compiler/Lir.fs | 52 +++++++++++++++++++ tests/FScript.Compiler.Tests/HirTests.fs | 18 +++++++ .../FScript.Compiler.Tests/Snapshots/add.hir | 14 +++++ .../FScript.Compiler.Tests/Snapshots/add.lir | 16 ++++++ 6 files changed, 104 insertions(+) create mode 100644 tests/FScript.Compiler.Tests/Snapshots/add.hir create mode 100644 tests/FScript.Compiler.Tests/Snapshots/add.lir diff --git a/CHANGELOG.md b/CHANGELOG.md index 297f746..a4c2239 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added a canonical LIR debug printer and checked-in deterministic HIR/LIR arithmetic snapshots. + - Preserved structured export names and source spans through resolved programs, HIR, LIR, and artifacts. - Added reachability-sensitive Go diagnostics for unsupported host externs without blocking unused descriptors. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index a78c499..5eab259 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -26,6 +26,8 @@ LIR is executable but not a Go AST. Lambda bodies are lifted into a deterministi The pass pipeline is HIR lowering, HIR validation, LIR lowering, `LirOptimization.optimize`, then LIR validation. The optimization pass is intentionally identity-only in version 1; its explicit boundary prevents later transformations from being hidden inside lowering or a backend. +HIR and LIR expose canonical debug printers with stable artifact-local IDs and normalized type rendering. The arithmetic vertical slice is checked in as paired snapshots so changes to symbol allocation, function lifting, captures, or operation selection are reviewed explicitly. + HIR validation rejects duplicate and unresolved symbols, invalid declaration spans, and schemes inconsistent with their values. LIR validation rechecks artifact-local references and spans, then verifies closure calls, branches and decision guards, primitive signatures, aggregate representations, declaration value types, intrinsic catalog membership, and complete target-import linkage. Artifact deserialization runs the same LIR validator, so malformed JSON cannot bypass compiler invariants. Lexical `let` is retained as a structured LIR node. Its symbol is visible only in the body, or in both the binding and body for recursive functions. The interpreter closes the recursive environment explicitly; Go emission declares the function slot before assigning its closure. `raise` becomes an explicit `Fail` node so consumers preserve evaluation order and attach their own stage-specific failure diagnostic. diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index 30059c3..747db14 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -131,6 +131,58 @@ module Lir = let symbolNumber (SymbolId value) = value let functionNumber (FunctionId value) = value + let format (program: Program) = + let lines = ResizeArray() + let typeName = Types.typeToString + let symbols values = values |> List.map (symbolNumber >> string) |> String.concat "," |> fun value -> "[" + value + "]" + let rec expression indent (value: Expression) = + let prefix = System.String(' ', indent) + let add label = lines.Add($"{prefix}{label} : {typeName value.Type}") + let children label values = add label; values |> List.iter (expression (indent + 2)) + match value.Node with + | Constant constant -> add $"const {constant}" + | Load id -> add $"load %%{symbolNumber id}" + | MakeClosure(id, captures) -> add $"make-closure @{functionNumber id} captures={symbols captures}" + | CallClosure(fn, argument) -> children "call-closure" [ fn; argument ] + | Branch(condition, whenTrue, whenFalse) -> children "branch" [ condition; whenTrue; whenFalse ] + | Operation(operation, arguments) -> children $"operation {operation}" arguments + | Tuple items -> children "tuple" items + | List items -> children "list" items + | Record fields -> children ("record " + (fields |> List.map fst |> String.concat ",")) (List.map snd fields) + | FieldGet(target, name) -> children $"field-get {name}" [ target ] + | SomeValue item -> children "some" [ item ] + | NoneValue -> add "none" + | Cons(head, tail) -> children "cons" [ head; tail ] + | Append(left, right) -> children "append" [ left; right ] + | Let(id, name, recursive, binding, body) -> children $"let %%{symbolNumber id} {name} recursive={recursive}" [ binding; body ] + | LetRecursiveGroup(bindings, body) -> children ("let-rec-group " + (bindings |> List.map (fun (id, name, _, _) -> $"%%{symbolNumber id}:{name}") |> String.concat ",")) ((bindings |> List.map (fun (_, _, _, item) -> item)) @ [ body ]) + | Fail item -> children "fail" [ item ] + | DecisionTree(target, cases) -> + add $"decision-tree cases={cases.Length}"; expression (indent + 2) target + cases |> List.iter (fun case -> case.Guard |> Option.iter (expression (indent + 2)); expression (indent + 2) case.Body) + | Range(first, last) -> children "range" [ first; last ] + | RecordUpdate(target, fields) -> children ("record-update " + (fields |> List.map fst |> String.concat ",")) (target :: List.map snd fields) + | RecordConvert(target, targetType) -> children $"record-convert {typeName targetType}" [ target ] + | IndexGet(target, index) -> children "index-get" [ target; index ] + | Interpolate parts -> children "interpolate" (parts |> List.choose (function TextPart _ -> None | ValuePart item -> Some item)) + | UnionCase(typeName, caseName, payload) -> children $"union {typeName}.{caseName}" (Option.toList payload) + | Map entries -> children "map" (entries |> List.collect (function KeyValue(key, item) -> [ key; item ] | Spread item -> [ item ])) + | Intrinsic name -> add $"intrinsic {name}" + | HostExternal(name, arity) -> add $"host-external {name}/{arity}" + | RuntimeContext -> add "runtime-context" + | TargetImport(modulePath, memberName) -> add $"target-import {modulePath}.{memberName}" + | ForEach(id, name, _, source, body) -> children $"for-each %%{symbolNumber id} {name}" [ source; body ] + | TypeToken tokenType -> add $"type-token {typeName tokenType}" + for fn in program.Functions do + lines.Add($"function @{functionNumber fn.Id} %%{symbolNumber fn.Parameter} {fn.Name} captures={symbols fn.Captures}: {typeName fn.ParameterType} -> {typeName fn.ReturnType}") + expression 2 fn.Body + for declaration in program.Declarations do + let export = declaration.Export |> Option.map (fun metadata -> $" export={metadata.Name}") |> Option.defaultValue "" + lines.Add($"decl %%{symbolNumber declaration.Id} {declaration.Name}{export}: {typeName declaration.Type}") + expression 2 declaration.Value + program.Entry |> Option.iter (fun entry -> lines.Add("entry"); expression 2 entry) + System.String.Join("\n", lines) + "\n" + module LirLowering = type private State = { mutable NextFunction: int diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index 031c365..86a746d 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -1,5 +1,6 @@ namespace FScript.Compiler.Tests +open System.IO open FScript.Compiler open FScript.Language open NUnit.Framework @@ -7,6 +8,11 @@ open NUnit.Framework [] type HirTests() = let span = Span.mk (Span.pos 1 0) (Span.pos 1 1) + let snapshot name = File.ReadAllText(Path.Combine(__SOURCE_DIRECTORY__, "Snapshots", name)).Replace("\r\n", "\n") + let compiledAdd () = + match Compiler.compile { Program = Parser.parseProgram "let add x y = x + y\nadd 1 2"; Externs = [] } with + | Ok value -> value + | Error diagnostics -> failwith (System.String.Join("; ", diagnostics |> List.map _.Message)) [] member _.``Lowers deterministic typed add program`` () = @@ -21,6 +27,18 @@ type HirTests() = Assert.That(actual, Does.Contain("entry")) Assert.That(Hir.format compiled.Hir, Is.EqualTo(actual)) + [] + member _.``Canonical HIR add snapshot is stable`` () = + let compiled = compiledAdd () + let declarations = compiled.Hir.Declarations |> List.filter (fun declaration -> declaration.Name = "add" || declaration.Name.StartsWith("$expression")) + Assert.That(Hir.format { compiled.Hir with Declarations = declarations }, Is.EqualTo(snapshot "add.hir")) + + [] + member _.``Canonical LIR add snapshot is stable`` () = + let compiled = compiledAdd () + let declarations = compiled.Lir.Declarations |> List.filter (fun declaration -> declaration.Name = "add" || declaration.Name.StartsWith("$expression")) + Assert.That(Lir.format { compiled.Lir with Declarations = declarations }, Is.EqualTo(snapshot "add.lir")) + [] member _.``HIR validation rejects a declaration scheme mismatch`` () = let value: Hir.Expression = { Type = TInt; Span = span; Node = Hir.Constant(Hir.Int 1L) } diff --git a/tests/FScript.Compiler.Tests/Snapshots/add.hir b/tests/FScript.Compiler.Tests/Snapshots/add.hir new file mode 100644 index 0000000..13fe6ae --- /dev/null +++ b/tests/FScript.Compiler.Tests/Snapshots/add.hir @@ -0,0 +1,14 @@ +decl %54 add: ('b -> ('b -> int)) + lambda %55 x: 'a : ('a -> ('a -> int)) + lambda %56 y: 'a : ('a -> int) + primitive IntAdd : int + ref %55 : 'a + ref %56 : 'a +decl %57 $expression57: int + apply : int + apply : (int -> int) + ref %54 : (int -> (int -> int)) + const int 1 : int + const int 2 : int +entry + ref %57 : int diff --git a/tests/FScript.Compiler.Tests/Snapshots/add.lir b/tests/FScript.Compiler.Tests/Snapshots/add.lir new file mode 100644 index 0000000..698fccf --- /dev/null +++ b/tests/FScript.Compiler.Tests/Snapshots/add.lir @@ -0,0 +1,16 @@ +function @0 %55 x captures=[]: 'a -> ('a -> int) + make-closure @1 captures=[55] : ('a -> int) +function @1 %56 y captures=[55]: 'a -> int + operation IntAdd : int + load %55 : 'a + load %56 : 'a +decl %54 add: ('a -> ('a -> int)) + make-closure @0 captures=[] : ('a -> ('a -> int)) +decl %57 $expression57: int + call-closure : int + call-closure : (int -> int) + load %54 : (int -> (int -> int)) + const Int 1L : int + const Int 2L : int +entry + load %57 : int From 5c3918b6582e43b257a988cac7444aa552686043 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 19:45:29 +0200 Subject: [PATCH 044/133] feat(go): preserve named record representations --- CHANGELOG.md | 2 + docs/architecture/typed-ir-and-backends.md | 4 +- docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Backend.Go/GoBackend.fs | 85 +++++++++-- src/FScript.Backend.Go/Runtime/patterns.go | 6 +- src/FScript.Backend.Go/Runtime/runtime.go | 142 +++++++++++++++--- src/FScript.Compiler/Hir.fs | 13 +- src/FScript.Compiler/HirLowering.fs | 45 +++++- src/FScript.Compiler/HirValidation.fs | 17 ++- src/FScript.Compiler/IrArtifact.fs | 33 +++- src/FScript.Compiler/Lir.fs | 51 ++++++- .../GoBackendTests.fs | 14 ++ tests/FScript.Compiler.Tests/HirTests.fs | 22 ++- .../FScript.Compiler.Tests/IrArtifactTests.fs | 9 ++ 14 files changed, 385 insertions(+), 59 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a4c2239..077ab51 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Preserved named type declarations through HIR, LIR, and artifacts and generated concrete Go structs for ordinary records. + - Added a canonical LIR debug printer and checked-in deterministic HIR/LIR arithmetic snapshots. - Preserved structured export names and source spans through resolved programs, HIR, LIR, and artifacts. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 5eab259..99d5a31 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -20,10 +20,12 @@ Validated LIR consumers are separate leaf assemblies. `FScript.Interpreter.Lir` ## IR boundaries -HIR is language-oriented and contains resolved artifact-local symbol identities, complete semantic types and schemes, source locations, structured export metadata, and explicit nodes for named/structural record conversions. Export metadata retains the exported source name and span through LIR and artifacts while the legacy Boolean remains available to compatibility consumers. Conversion evidence is produced by inference, not rediscovered from backend representations. HIR contains no Go types. +HIR is language-oriented and contains resolved artifact-local symbol identities, ordinary and imported type declarations, complete semantic types and schemes, source locations, structured export metadata, and explicit nodes for named/structural record conversions. Named record and union references remain nominal in HIR and LIR; their declaration shapes are carried separately and serialized in artifacts. Export metadata retains the exported source name and span through LIR and artifacts while the legacy Boolean remains available to compatibility consumers. Conversion evidence is produced by inference, not rediscovered from backend representations. HIR contains no Go types. LIR is executable but not a Go AST. Lambda bodies are lifted into a deterministic program-level function table. Closure creation references a function ID and carries its canonical ordered set of captured symbol IDs, alongside resolved primitive operations, aggregate construction/projection, and structured control flow. Validation recomputes each lifted function's free-symbol set and rejects unknown function IDs or forged, missing, duplicate, or unordered captures. Pattern matching becomes deterministic tests and projections before reaching a backend. Semantic types remain alongside selected runtime representations for validation and diagnostics. +The Go backend selects concrete generated structs for ordinary named records, with deterministic synthetic field identifiers and FScript-name tags. Structural records remain runtime maps, and explicit LIR conversion nodes copy fields between the representations while evaluating their source once. Runtime record projection, patterns, formatting, and equality reflect only tagged generated fields, so nominal representation does not alter FScript structural behavior. Imported record adapters remain isolated at target-call boundaries. + The pass pipeline is HIR lowering, HIR validation, LIR lowering, `LirOptimization.optimize`, then LIR validation. The optimization pass is intentionally identity-only in version 1; its explicit boundary prevents later transformations from being hidden inside lowering or a backend. HIR and LIR expose canonical debug printers with stable artifact-local IDs and normalized type rendering. The arithmetic vertical slice is checked in as paired snapshots so changes to symbol allocation, function lifting, captures, or operation selection are reviewed explicitly. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index c0cc4a5..f0d17f8 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -29,6 +29,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Tuples and lists preserve element order. Options distinguish `None` from `Some value`. - Maps currently accept string keys through inference and have deterministic ordered-map behavior. Runtime support for integer map keys is legacy representation detail, not permission for newly compiled programs. - Declared records retain declaration identity for named annotations. Structural record annotations retain field-shape compatibility and may accept additional fields where current inference permits it. Inference records coercion evidence at the accepting application; HIR and LIR retain an explicit conversion node so backends convert representations without changing compatibility rules. +- HIR, LIR, and version-1 artifacts carry ordinary record and union declarations separately from nominal references. Generated Go uses concrete structs for ordinary named records, copies record updates, and performs explicit structural/named conversions; projection, patterns, formatting, and recursive equality remain shape-compatible. - Discriminated unions retain their declared type name, case name, and optional payload. Recursive types require `type rec`; mutually recursive type-definition graphs are rejected. - `typeof` produces an FScript type token and `nameof` produces the resolved source name. - Compiled local pattern bindings evaluate their source once and use the ordinary match-failure path. Compiled `for` evaluates its list once, executes bodies from first to last, discards their values, and returns unit. Compiled `typeof` retains the resolved declared record or union shape in the artifact instead of performing a name lookup at execution time. diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index b32890e..41328ce 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -11,9 +11,23 @@ type GeneratedFile = { RelativePath: string; Content: string } module GoBackend = let private currentImportedRecords = AsyncLocal option>() + let private currentTypes = AsyncLocal option>() let private currentFunctions = AsyncLocal>() + let private currentSymbolTypes = AsyncLocal>() let private tryImportedRecord name = currentImportedRecords.Value |> Option.bind (Map.tryFind name) + let private tryDeclaredType name = + currentTypes.Value |> Option.bind (Map.tryFind name) + let private declaredTypeName (declaration: Lir.TypeDeclaration) = $"record{Lir.symbolNumber declaration.Id}" + let private tryRecordDeclaration name = + tryDeclaredType name + |> Option.bind (fun declaration -> match declaration.Import, declaration.Definition with None, TRecord fields -> Some(declaration, fields) | _ -> None) + let private recordFieldName fields name = + fields |> Map.toList |> List.findIndex (fst >> (=) name) |> sprintf "Field%d" + let private effectiveType (expression: Lir.Expression) = + match expression.Node with + | Lir.Load id -> currentSymbolTypes.Value |> Map.tryFind id |> Option.defaultValue expression.Type + | _ -> expression.Type let private indent (value: string) = value.Replace("\n", "\n\t") let private runtimeTemplate fileName = @@ -46,6 +60,7 @@ module GoBackend = | TRecord _ -> "map[string]any" | TNamed "Environment" -> "map[string]any" | TNamed name when tryImportedRecord name |> Option.isSome -> "map[string]any" + | TNamed name when tryRecordDeclaration name |> Option.isSome -> tryRecordDeclaration name |> Option.get |> fst |> declaredTypeName | TUnion _ | TNamed _ -> "fsruntime.Union" | TMap _ -> "fsruntime.OrderedMap" | value -> invalidArg "value" $"Unsupported Go representation for {Types.typeToString value}." @@ -87,6 +102,7 @@ module GoBackend = | TOption _ -> $"fsruntime.AsOption({code})" | TNamed "Environment" -> $"fsruntime.Record({code})" | TNamed name when tryImportedRecord name |> Option.isSome -> $"fsruntime.Record({code})" + | TNamed name when tryRecordDeclaration name |> Option.isSome -> $"{code}.({goType valueType})" | TUnion _ | TNamed _ -> $"fsruntime.AsUnion({code})" | TMap _ -> $"fsruntime.AsMap({code})" | TFun _ -> $"fsruntime.AsClosure({code})" @@ -291,12 +307,24 @@ module GoBackend = let joined = String.concat ", " values $"[]any{{{joined}}}") | Lir.Record fields -> - fields |> List.map (fun (name, value) -> emitExpression environment value |> Result.map (fun code -> $"{System.Text.Json.JsonSerializer.Serialize(name)}: {code}")) + let declaration = match expression.Type with TNamed name -> tryRecordDeclaration name | _ -> None + fields |> List.map (fun (name, value) -> emitExpression environment value |> Result.map (fun code -> name, code)) |> collect |> Result.map (fun values -> - let joined = String.concat ", " values - $"map[string]any{{{joined}}}") + match declaration with + | Some(typeDeclaration, fieldTypes) -> + let joined = values |> List.map (fun (name, code) -> $"{recordFieldName fieldTypes name}: {code}") |> String.concat ", " + $"{declaredTypeName typeDeclaration}{{{joined}}}" + | None -> + let joined = values |> List.map (fun (name, code) -> $"{System.Text.Json.JsonSerializer.Serialize(name)}: {code}") |> String.concat ", " + $"map[string]any{{{joined}}}") | Lir.FieldGet(target, name) -> - emitExpression environment target |> Result.map (fun code -> $"({code})[{System.Text.Json.JsonSerializer.Serialize(name)}]") + emitExpression environment target |> Result.map (fun code -> + match effectiveType target with + | TNamed typeName -> + match tryRecordDeclaration typeName with + | Some(_, fields) -> $"({code}).{recordFieldName fields name}" + | None -> $"({code})[{System.Text.Json.JsonSerializer.Serialize(name)}]" + | _ -> $"({code})[{System.Text.Json.JsonSerializer.Serialize(name)}]") | Lir.SomeValue item -> emitExpression environment item |> Result.map (fun code -> $"fsruntime.Some({code})") | Lir.NoneValue -> Ok "fsruntime.None()" | Lir.Cons(head, tail) -> @@ -355,12 +383,35 @@ module GoBackend = | Error error, _ | _, Error error -> Error error | Lir.RecordUpdate(target, fields) -> match emitExpression environment target, - (fields |> List.map (fun (name, item) -> emitExpression environment item |> Result.map (fun code -> System.Text.Json.JsonSerializer.Serialize(name) + ": " + code)) |> collect) with + (fields |> List.map (fun (name, item) -> emitExpression environment item |> Result.map (fun code -> name, code)) |> collect) with | Ok targetCode, Ok values -> - let joined = String.concat ", " values - Ok $"fsruntime.UpdateRecord({targetCode}, map[string]any{{{joined}}})" + match expression.Type with + | TNamed name -> + match tryRecordDeclaration name with + | Some(declaration, fieldTypes) -> + let assignments = values |> List.map (fun (fieldName, code) -> $"updated.{recordFieldName fieldTypes fieldName} = {code}") |> String.concat "\n\t" + Ok $"func() {declaredTypeName declaration} {{\n\tupdated := {indent targetCode}\n\t{assignments}\n\treturn updated\n}}()" + | None -> + let joined = values |> List.map (fun (fieldName, code) -> System.Text.Json.JsonSerializer.Serialize(fieldName) + ": " + code) |> String.concat ", " + Ok $"fsruntime.UpdateRecord({targetCode}, map[string]any{{{joined}}})" + | _ -> + let joined = values |> List.map (fun (fieldName, code) -> System.Text.Json.JsonSerializer.Serialize(fieldName) + ": " + code) |> String.concat ", " + Ok $"fsruntime.UpdateRecord({targetCode}, map[string]any{{{joined}}})" | Error error, _ | _, Error error -> Error error - | Lir.RecordConvert(target, _) -> emitExpression environment target + | Lir.RecordConvert(target, targetType) -> + emitExpression environment target |> Result.map (fun targetCode -> + match targetType with + | TNamed name -> + match tryRecordDeclaration name with + | Some(declaration, fields) -> + let assignments = + fields |> Map.toList |> List.map (fun (fieldName, fieldType) -> + let raw = $"recordFields[{System.Text.Json.JsonSerializer.Serialize(fieldName)}]" + $"{recordFieldName fields fieldName}: {cast fieldType raw}") |> String.concat ", " + $"func() {declaredTypeName declaration} {{\n\trecordValue := {indent targetCode}\n\trecordFields := fsruntime.Record(recordValue)\n\treturn {declaredTypeName declaration}{{{assignments}}}\n}}()" + | None -> $"fsruntime.Record({targetCode})" + | TRecord _ -> $"fsruntime.Record({targetCode})" + | _ -> targetCode) | Lir.IndexGet(target, index) -> match emitExpression environment target, emitExpression environment index with | Ok targetCode, Ok indexCode -> Ok $"fsruntime.Index({targetCode}, {indexCode})" @@ -499,7 +550,9 @@ func Environment() map[string]any { let private generateCore options (program: Lir.Program) : Result = try currentImportedRecords.Value <- Some(program.ImportedRecords |> List.map (fun record -> record.Name, record) |> Map.ofList) + currentTypes.Value <- Some(program.Types |> List.map (fun declaration -> declaration.Name, declaration) |> Map.ofList) currentFunctions.Value <- program.Functions |> List.map (fun fn -> fn.Id, fn) |> Map.ofList + currentSymbolTypes.Value <- program.Declarations |> List.map (fun declaration -> declaration.Id, declaration.Type) |> Map.ofList let targetImports = program.Declarations |> List.choose (fun declaration -> match declaration.Value.Node with Lir.TargetImport(modulePath, memberName) -> Some(modulePath, memberName, declaration.Type, declaration.Span) | _ -> None) @@ -516,6 +569,17 @@ func Environment() map[string]any { program.Declarations |> List.fold (fun env declaration -> env |> Map.add declaration.Id (identifier "value" declaration.Id)) Map.empty let declarations = StringBuilder() + let typeDeclarations = StringBuilder() + for declaration in program.Types do + match declaration.Import, declaration.Definition with + | None, TRecord fields -> + typeDeclarations.AppendLine($"type {declaredTypeName declaration} struct {{") |> ignore + let fieldTypes = fields |> Map.toList |> List.map (snd >> goType) + let typeWidth = fieldTypes |> List.map _.Length |> List.max + for fieldIndex, (fieldName, fieldType) in fields |> Map.toList |> List.indexed do + typeDeclarations.AppendLine($"\tField{fieldIndex} {(goType fieldType).PadRight(typeWidth)} `fscript:{System.Text.Json.JsonSerializer.Serialize(fieldName)}`") |> ignore + typeDeclarations.AppendLine("}") |> ignore + | _ -> () let initializers = StringBuilder() let mutable error = None for declaration in program.Declarations do @@ -543,7 +607,8 @@ func Environment() map[string]any { ([ yield! targetImports |> List.map (fun (modulePath, _, _, _) -> modulePath) yield! program.ImportedRecords |> List.map _.ModulePath ] |> List.distinct |> List.sort |> List.map (fun path -> $"\t{importAlias path} {System.Text.Json.JsonSerializer.Serialize(path)}")) |> String.concat "\n" - let programSource = $"package fscriptprogram\n\nimport (\n{importLines}\n)\n\n{declarations}\nfunc init() {{\n{initializers}}}\n\nfunc Run() any {{ return {entryCode} }}\n" + let typeSource = if typeDeclarations.Length = 0 then "" else typeDeclarations.ToString() + "\n" + let programSource = $"package fscriptprogram\n\nimport (\n{importLines}\n)\n\n{typeSource}{declarations}\nfunc init() {{\n{initializers}}}\n\nfunc Run() any {{ return {entryCode} }}\n" let printExpression = match program.Entry with | Some { Type = TUnit } -> "_ = fsruntime.Format(fscriptprogram.Run()); fmt.Println(\"()\")" @@ -561,7 +626,9 @@ func Environment() map[string]any { let generate options (program: Lir.Program) : Result = currentImportedRecords.Value <- Some(program.ImportedRecords |> List.map (fun record -> record.Name, record) |> Map.ofList) + currentTypes.Value <- Some(program.Types |> List.map (fun declaration -> declaration.Name, declaration) |> Map.ofList) currentFunctions.Value <- program.Functions |> List.map (fun fn -> fn.Id, fn) |> Map.ofList + currentSymbolTypes.Value <- program.Declarations |> List.map (fun declaration -> declaration.Id, declaration.Type) |> Map.ofList let rec containsTask = function | TTask _ -> true | TFun(argument, result) | TMap(argument, result) -> containsTask argument || containsTask result diff --git a/src/FScript.Backend.Go/Runtime/patterns.go b/src/FScript.Backend.Go/Runtime/patterns.go index 0532daf..bcc52bf 100644 --- a/src/FScript.Backend.Go/Runtime/patterns.go +++ b/src/FScript.Backend.Go/Runtime/patterns.go @@ -90,7 +90,7 @@ func matchRuntimePattern(value any, pattern Pattern) (map[int]any, bool) { } return bindings, true case "record": - values, ok := value.(map[string]any) + values, ok := TryRecord(value) if !ok { return nil, false } @@ -210,12 +210,12 @@ func TypeMatches(value any, typeName string) bool { return ok } if strings.HasPrefix(typeName, "{") { - _, ok := value.(map[string]any) + _, ok := TryRecord(value) return ok } if item, ok := value.(Union); ok { return item.TypeName == typeName } - _, record := value.(map[string]any) + _, record := TryRecord(value) return record } diff --git a/src/FScript.Backend.Go/Runtime/runtime.go b/src/FScript.Backend.Go/Runtime/runtime.go index ac4fb42..a535676 100644 --- a/src/FScript.Backend.Go/Runtime/runtime.go +++ b/src/FScript.Backend.Go/Runtime/runtime.go @@ -31,21 +31,50 @@ type MapPart struct { Map OrderedMap } -func Unit(value any) struct{} { return struct{}{} } -func Int(value any) int64 { return value.(int64) } -func Float(value any) float64 { return value.(float64) } -func Bool(value any) bool { return value.(bool) } -func String(value any) string { return value.(string) } -func List(value any) []any { return value.([]any) } -func AsTuple(value any) Tuple { return value.(Tuple) } -func Record(value any) map[string]any { return value.(map[string]any) } -func AsOption(value any) Option { return value.(Option) } -func AsUnion(value any) Union { return value.(Union) } -func AsMap(value any) OrderedMap { return value.(OrderedMap) } -func Some(value any) Option { return Option{HasValue: true, Value: value} } -func None() Option { return Option{} } -func IsSome(value any) bool { return AsOption(value).HasValue } -func OptionValue(value any) any { return AsOption(value).Value } +func Unit(value any) struct{} { return struct{}{} } +func Int(value any) int64 { return value.(int64) } +func Float(value any) float64 { return value.(float64) } +func Bool(value any) bool { return value.(bool) } +func String(value any) string { return value.(string) } +func List(value any) []any { return value.([]any) } +func AsTuple(value any) Tuple { return value.(Tuple) } +func TryRecord(value any) (map[string]any, bool) { + if record, ok := value.(map[string]any); ok { + return record, true + } + reflected := reflect.ValueOf(value) + if reflected.Kind() == reflect.Pointer { + reflected = reflected.Elem() + } + if reflected.Kind() != reflect.Struct { + return nil, false + } + result := map[string]any{} + for index := 0; index < reflected.NumField(); index++ { + field := reflected.Type().Field(index) + if name := field.Tag.Get("fscript"); name != "" { + result[name] = reflected.Field(index).Interface() + } + } + if len(result) == 0 { + return nil, false + } + return result, true +} +func Record(value any) map[string]any { + record, ok := TryRecord(value) + if !ok { + panic("Record access requires a record") + } + return record +} +func AsOption(value any) Option { return value.(Option) } +func AsUnion(value any) Union { return value.(Union) } +func AsMap(value any) OrderedMap { return value.(OrderedMap) } +func Some(value any) Option { return Option{HasValue: true, Value: value} } +func None() Option { return Option{} } +func IsSome(value any) bool { return AsOption(value).HasValue } +func OptionValue(value any) any { return AsOption(value).Value } func MakeUnion(typeName string, caseName string) Union { return Union{TypeName: typeName, CaseName: caseName} } @@ -56,10 +85,69 @@ func IsUnion(value any, typeName string, caseName string) bool { item := AsUnion(value) return item.TypeName == typeName && item.CaseName == caseName } -func UnionHasValue(value any) bool { return AsUnion(value).HasValue } -func UnionValue(value any) any { return AsUnion(value).Value } -func HasField(value any, name string) bool { _, ok := Record(value)[name]; return ok } -func Equal(left any, right any) bool { return reflect.DeepEqual(left, right) } +func UnionHasValue(value any) bool { return AsUnion(value).HasValue } +func UnionValue(value any) any { return AsUnion(value).Value } +func HasField(value any, name string) bool { + record, ok := TryRecord(value) + if !ok { + return false + } + _, found := record[name] + return found +} +func normalizeStructural(value any) any { + reflected := reflect.ValueOf(value) + if reflected.IsValid() && (reflected.Kind() == reflect.Struct || (reflected.Kind() == reflect.Pointer && !reflected.IsNil() && reflected.Elem().Kind() == reflect.Struct)) { + if record, ok := TryRecord(value); ok { + result := map[string]any{} + for name, field := range record { + result[name] = normalizeStructural(field) + } + return result + } + } + switch item := value.(type) { + case []any: + result := make([]any, len(item)) + for index, element := range item { + result[index] = normalizeStructural(element) + } + return result + case Tuple: + result := make(Tuple, len(item)) + for index, element := range item { + result[index] = normalizeStructural(element) + } + return result + case map[string]any: + result := map[string]any{} + for name, field := range item { + result[name] = normalizeStructural(field) + } + return result + case Option: + if item.HasValue { + item.Value = normalizeStructural(item.Value) + } + return item + case Union: + if item.HasValue { + item.Value = normalizeStructural(item.Value) + } + return item + case OrderedMap: + entries := make([]MapEntry, len(item.Entries)) + for index, entry := range item.Entries { + entries[index] = MapEntry{Key: normalizeStructural(entry.Key), Value: normalizeStructural(entry.Value)} + } + return OrderedMap{Entries: entries} + default: + return value + } +} +func Equal(left any, right any) bool { + return reflect.DeepEqual(normalizeStructural(left), normalizeStructural(right)) +} func Cons(head any, tail []any) []any { return append([]any{head}, tail...) } func Append(left []any, right []any) []any { return append(append([]any{}, left...), right...) } func Tail(value any) []any { return append([]any{}, List(value)[1:]...) } @@ -279,7 +367,23 @@ func Format(value any) string { parts[index] = Format(value) } return "(" + strings.Join(parts, ", ") + ")" + case map[string]any: + names := make([]string, 0, len(item)) + for name := range item { + names = append(names, name) + } + sort.Strings(names) + parts := make([]string, len(names)) + for index, name := range names { + parts[index] = name + " = " + Format(item[name]) + } + return "{ " + strings.Join(parts, "; ") + " }" default: + if reflected := reflect.ValueOf(value); reflected.IsValid() && (reflected.Kind() == reflect.Struct || (reflected.Kind() == reflect.Pointer && !reflected.IsNil() && reflected.Elem().Kind() == reflect.Struct)) { + if record, ok := TryRecord(value); ok { + return Format(record) + } + } return fmt.Sprint(value) } } diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs index b12dd61..1eb3a8a 100644 --- a/src/FScript.Compiler/Hir.fs +++ b/src/FScript.Compiler/Hir.fs @@ -123,8 +123,17 @@ module Hir = Fields: (string * Type) list Span: Span } + type TypeDeclaration = + { Id: SymbolId + Name: string + IsRecursive: bool + Definition: Type + Import: ImportedRecord option + Span: Span } + type Program = - { ImportedRecords: ImportedRecord list + { Types: TypeDeclaration list + ImportedRecords: ImportedRecord list Declarations: Declaration list Entry: Expression option } @@ -212,6 +221,8 @@ module Hir = lines.Add(typed (sprintf "for %%%d %s" (symbolNumber id) name)); expression (indent + 2) source; expression (indent + 2) body | TypeToken tokenType -> lines.Add(typed $"typeof {formatType tokenType}") + for typeDeclaration in program.Types do + lines.Add(sprintf "type %%%d %s%s = %s" (symbolNumber typeDeclaration.Id) typeDeclaration.Name (if typeDeclaration.IsRecursive then " rec" else "") (formatType typeDeclaration.Definition)) for declaration in program.Declarations do let export = declaration.Export |> Option.map (fun metadata -> $" export={metadata.Name}") |> Option.defaultValue "" lines.Add(sprintf "decl %%%d %s%s: %s" (symbolNumber declaration.Id) declaration.Name export (formatType (let (Forall(_, value)) = declaration.Scheme in value))) diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index d1e430f..df656e4 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -24,9 +24,18 @@ module HirLowering = | TMap(key, value) -> TMap(representationType state key, representationType state value) | TTask item -> TTask(representationType state item) | TRecord fields -> TRecord(fields |> Map.map (fun _ -> representationType state)) - | TNamed name -> state.TypeDefinitions |> Map.tryFind name |> Option.defaultValue (TNamed name) + | TNamed name -> TNamed name | value -> value + let private resolvedDefinition state = function + | TNamed name as named -> state.TypeDefinitions |> Map.tryFind name |> Option.defaultValue named + | value -> value + + let private recordFields state value = + match resolvedDefinition state value with + | TRecord fields -> fields + | _ -> Map.empty + let private expressionType state expression fallback = state.ExpressionTypes |> Map.tryFind expression @@ -94,7 +103,7 @@ module HirLowering = let bindings = bindingMaps |> List.fold (Map.fold (fun env key value -> Map.add key value env)) Map.empty make (Hir.TuplePattern patterns), bindings) | PRecord(fields, _) -> - let types = match expected with TRecord values -> values | _ -> Map.empty + let types = recordFields state expected fields |> List.map (fun (name, item) -> lowerPattern state (types |> Map.tryFind name |> Option.defaultValue (Types.freshVar())) item |> Result.map (fun (value, bindings) -> (name, value), bindings)) |> collectResults |> Result.map (fun values -> let fields, bindingMaps = List.unzip values @@ -216,7 +225,7 @@ module HirLowering = let itemType = match inferred with TList value -> value | _ -> Types.freshVar() items |> List.map (lowerExpression state environment itemType) |> collectResults |> Result.map (Hir.List >> make) | ERecord(fields, _) | EStructuralRecord(fields, _) -> - let fieldTypes = match inferred with TRecord values -> values | _ -> Map.empty + let fieldTypes = recordFields state inferred fields |> List.map (fun (name, item) -> lowerExpression state environment (fieldTypes |> Map.tryFind name |> Option.defaultValue (Types.freshVar())) item |> Result.map (fun value -> name, value)) |> collectResults |> Result.map (Hir.Record >> make) | EFieldGet(target, name, _) -> @@ -293,11 +302,19 @@ module HirLowering = match lowerExpression state environment TInt startValue, lowerExpression state environment TInt endValue with | Ok startItem, Ok endItem -> Ok(make (Hir.Range(startItem, endItem))) | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e - | ERecordUpdate(target, fields, _) | EStructuralRecordUpdate(target, fields, _) -> + | ERecordUpdate(target, fields, _) -> + let targetType = expressionType state target inferred + match lowerExpression state environment targetType target with + | Error errors -> Error errors + | Ok loweredTarget -> + let fieldTypes = recordFields state targetType + fields |> List.map (fun (name, item) -> lowerExpression state environment (fieldTypes |> Map.tryFind name |> Option.defaultValue (Types.freshVar())) item |> Result.map (fun value -> name, value)) + |> collectResults |> Result.map (fun loweredFields -> ({ Type = targetType; Span = span; Node = Hir.RecordUpdate(loweredTarget, loweredFields) }: Hir.Expression)) + | EStructuralRecordUpdate(target, fields, _) -> match lowerExpression state environment inferred target with | Error errors -> Error errors | Ok loweredTarget -> - let fieldTypes = match inferred with TRecord values -> values | _ -> Map.empty + let fieldTypes = recordFields state inferred fields |> List.map (fun (name, item) -> lowerExpression state environment (fieldTypes |> Map.tryFind name |> Option.defaultValue (Types.freshVar())) item |> Result.map (fun value -> name, value)) |> collectResults |> Result.map (fun loweredFields -> make (Hir.RecordUpdate(loweredTarget, loweredFields))) | EIndexGet(target, index, _) -> @@ -374,6 +391,10 @@ module HirLowering = unionDefinitions |> List.collect (fun (definition, _) -> let cases = definition.Cases |> List.map (fun (name, payload) -> name, Option.map typeFromRef payload) |> Map.ofList definition.Cases |> List.map (fun (caseName, _) -> caseName, (definition.Name, cases.[caseName]))) |> Map.ofList + let typeSymbols = + typeDefinitions + |> List.map (fun definition -> definition.Name, newSymbol state) + |> Map.ofList let constructorSymbols = unionDefinitions |> List.collect (fun (definition, _) -> definition.Cases |> List.map (fun (caseName, _) -> definition.Name, caseName, newSymbol state)) let topLevel = @@ -406,7 +427,8 @@ module HirLowering = | UnsupportedHostExtern -> Hir.HostExternal(external.Name, external.Arity) ({ Id = externSymbols.[external.Name]; Name = external.Name; Scheme = external.Scheme; IsRecursive = false; IsExported = false; Export = None; Span = span; Value = { Type = valueType; Span = span; Node = node } }: Hir.Declaration)) builtinDeclarations @ externalDeclarations @ (constructorSymbols |> List.map (fun (typeName, caseName, id) -> - let definition, unionType = unionDefinitions |> List.find (fun (definition, _) -> definition.Name = typeName) + let definition, _ = unionDefinitions |> List.find (fun (definition, _) -> definition.Name = typeName) + let unionType = TNamed typeName let payloadType = definition.Cases |> List.find (fst >> (=) caseName) |> snd |> Option.map typeFromRef let value = match payloadType with @@ -512,5 +534,14 @@ module HirLowering = ({ Name = definition.Name; ModulePath = import.ModulePath; MemberName = import.MemberName; IsUnion = not definition.Cases.IsEmpty Fields = definition.Fields |> List.map (fun (name, fieldType) -> name, typeFromRef fieldType) Span = definition.Span }: Hir.ImportedRecord))) - if errors.IsEmpty then Ok { ImportedRecords = importedRecords; Declarations = declarations; Entry = entry } + let hirTypes = + typeDefinitions + |> List.map (fun definition -> + ({ Id = typeSymbols.[definition.Name] + Name = definition.Name + IsRecursive = definition.IsRecursive + Definition = state.TypeDefinitions.[definition.Name] + Import = importedRecords |> List.tryFind (fun imported -> imported.Name = definition.Name) + Span = definition.Span }: Hir.TypeDeclaration)) + if errors.IsEmpty then Ok { Types = hirTypes; ImportedRecords = importedRecords; Declarations = declarations; Entry = entry } else Error errors diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index 82d0ba8..0bfff98 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -22,11 +22,24 @@ module HirValidation = let validate (program: Hir.Program) = let declarationIds = program.Declarations |> List.map (fun value -> value.Id) - let duplicateIds = declarationIds |> List.countBy id |> List.filter (fun (_, count) -> count > 1) + let typeIds = program.Types |> List.map _.Id + let allIds = typeIds @ declarationIds + let duplicateIds = allIds |> List.countBy id |> List.filter (fun (_, count) -> count > 1) + let invalidTypes = + (program.Types |> List.map _.Name |> List.distinct |> List.length) <> program.Types.Length + || program.Types |> List.exists (fun declaration -> + not (validSpan declaration.Span) + || match declaration.Definition with + | TRecord _ -> false + | TUnion(name, _) -> name <> declaration.Name + | _ -> true + || declaration.Import |> Option.exists (fun imported -> imported.Name <> declaration.Name)) let invalidImports = program.ImportedRecords |> List.exists (fun record -> System.String.IsNullOrWhiteSpace(record.ModulePath) || System.String.IsNullOrWhiteSpace(record.MemberName)) || (program.ImportedRecords |> List.map _.Name |> List.distinct |> List.length) <> program.ImportedRecords.Length - if invalidImports then + if invalidTypes then + Error [ Diagnostic.hir "FSH1107" "HIR type declarations must be unique, well-formed, and consistently linked." None ] + elif invalidImports then Error [ Diagnostic.hir "FSH1105" "HIR imported record linkage must be complete and uniquely named." None ] elif not duplicateIds.IsEmpty then Error [ Diagnostic.hir "FSH1101" "HIR declaration IDs must be unique." None ] diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index eb82d46..44bbe95 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -19,7 +19,8 @@ module IrArtifact = let private sourceMetadata (program: Lir.Program) = let paths = - [ yield! program.ImportedRecords |> List.choose (fun declaration -> declaration.Span.Start.File) + [ yield! program.Types |> List.choose (fun declaration -> declaration.Span.Start.File) + yield! program.ImportedRecords |> List.choose (fun declaration -> declaration.Span.Start.File) yield! program.Functions |> List.choose (fun fn -> fn.Span.Start.File) yield! program.Declarations |> List.choose (fun declaration -> declaration.Span.Start.File) match program.Entry with Some entry -> yield! entry.Span.Start.File |> Option.toList | None -> () ] @@ -257,7 +258,20 @@ module IrArtifact = writer.WriteEndArray(); writer.WriteStartArray("dependencies") dependencies program |> List.iter writer.WriteStringValue writer.WriteEndArray() - writer.WriteStartObject("program"); writer.WriteStartArray("importedRecords") + writer.WriteStartObject("program"); writer.WriteStartArray("types") + for declaration in program.Types do + writer.WriteStartObject(); writer.WriteNumber("id", Lir.symbolNumber declaration.Id); writer.WriteString("name", declaration.Name) + writer.WriteBoolean("recursive", declaration.IsRecursive); writer.WritePropertyName("definition"); writeType writer declaration.Definition + match declaration.Import with + | Some imported -> + writer.WriteStartObject("import"); writer.WriteString("name", imported.Name); writer.WriteString("modulePath", imported.ModulePath) + writer.WriteString("memberName", imported.MemberName); writer.WriteBoolean("union", imported.IsUnion); writer.WriteStartArray("fields") + for name, fieldType in imported.Fields do + writer.WriteStartObject(); writer.WriteString("name", name); writer.WritePropertyName("type"); writeType writer fieldType; writer.WriteEndObject() + writer.WriteEndArray(); writer.WritePropertyName("span"); writeSpan writer imported.Span; writer.WriteEndObject() + | None -> writer.WriteNull("import") + writer.WritePropertyName("span"); writeSpan writer declaration.Span; writer.WriteEndObject() + writer.WriteEndArray(); writer.WriteStartArray("importedRecords") for record in program.ImportedRecords do writer.WriteStartObject(); writer.WriteString("name", record.Name); writer.WriteString("modulePath", record.ModulePath); writer.WriteString("memberName", record.MemberName); writer.WriteBoolean("union", record.IsUnion) writer.WriteStartArray("fields") @@ -448,10 +462,18 @@ module IrArtifact = Error [ { Code = "FSA5005"; Stage = Artifact; Message = "IR artifact metadata is invalid or non-canonical."; Span = None } ] else let programElement = property "program" root - let importedRecords = (property "importedRecords" programElement).EnumerateArray() |> Seq.map (fun item -> + let readImportedRecord (item: JsonElement) = ({ Name = stringProperty "name" item; ModulePath = stringProperty "modulePath" item; MemberName = stringProperty "memberName" item; IsUnion = (property "union" item).GetBoolean() Fields = (property "fields" item).EnumerateArray() |> Seq.map (fun field -> stringProperty "name" field, readType (property "type" field)) |> Seq.toList - Span = readSpan (property "span" item) }: Lir.ImportedRecord)) |> Seq.toList + Span = readSpan (property "span" item) }: Lir.ImportedRecord) + let importedRecords = (property "importedRecords" programElement).EnumerateArray() |> Seq.map (fun item -> + readImportedRecord item) |> Seq.toList + let types = (property "types" programElement).EnumerateArray() |> Seq.map (fun item -> + let importElement = property "import" item + ({ Id = Lir.SymbolId((property "id" item).GetInt32()); Name = stringProperty "name" item + IsRecursive = (property "recursive" item).GetBoolean(); Definition = readType (property "definition" item) + Import = if importElement.ValueKind = JsonValueKind.Null then None else Some(readImportedRecord importElement) + Span = readSpan (property "span" item) }: Lir.TypeDeclaration)) |> Seq.toList let declarations = (property "declarations" programElement).EnumerateArray() |> Seq.map (fun (item: JsonElement) -> let exportElement = property "export" item let export = @@ -471,7 +493,8 @@ module IrArtifact = Body = readExpression (property "body" item) }: Lir.Function)) |> Seq.toList let entryElement = property "entry" programElement let program : Lir.Program = - { ImportedRecords = importedRecords + { Types = types + ImportedRecords = importedRecords Functions = functions Declarations = declarations Entry = if entryElement.ValueKind = JsonValueKind.Null then None else Some(readExpression entryElement) } diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index 747db14..99a71d7 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -122,8 +122,17 @@ module Lir = Fields: (string * Type) list Span: Span } + type TypeDeclaration = + { Id: SymbolId + Name: string + IsRecursive: bool + Definition: Type + Import: ImportedRecord option + Span: Span } + type Program = - { ImportedRecords: ImportedRecord list + { Types: TypeDeclaration list + ImportedRecords: ImportedRecord list Functions: Function list Declarations: Declaration list Entry: Expression option } @@ -173,6 +182,8 @@ module Lir = | TargetImport(modulePath, memberName) -> add $"target-import {modulePath}.{memberName}" | ForEach(id, name, _, source, body) -> children $"for-each %%{symbolNumber id} {name}" [ source; body ] | TypeToken tokenType -> add $"type-token {typeName tokenType}" + for typeDeclaration in program.Types do + lines.Add($"type %%{symbolNumber typeDeclaration.Id} {typeDeclaration.Name}: {typeName typeDeclaration.Definition}") for fn in program.Functions do lines.Add($"function @{functionNumber fn.Id} %%{symbolNumber fn.Parameter} {fn.Name} captures={symbols fn.Captures}: {typeName fn.ParameterType} -> {typeName fn.ReturnType}") expression 2 fn.Body @@ -338,6 +349,14 @@ module LirLowering = |> List.map (fun value -> ({ Name = value.Name; ModulePath = value.ModulePath; MemberName = value.MemberName; IsUnion = value.IsUnion Fields = value.Fields; Span = value.Span }: Lir.ImportedRecord)) + Types = + program.Types + |> List.map (fun value -> + ({ Id = symbol value.Id; Name = value.Name; IsRecursive = value.IsRecursive; Definition = value.Definition + Import = value.Import |> Option.map (fun imported -> + { Name = imported.Name; ModulePath = imported.ModulePath; MemberName = imported.MemberName + IsUnion = imported.IsUnion; Fields = imported.Fields; Span = imported.Span }) + Span = value.Span }: Lir.TypeDeclaration)) Functions = state.Functions |> Seq.sortBy (_.Id >> Lir.functionNumber) |> Seq.toList Declarations = declarations Entry = entry } @@ -362,12 +381,28 @@ module LirValidation = let validate (program: Lir.Program) = let ids = program.Declarations |> List.map _.Id + let typeIds = program.Types |> List.map _.Id + let typeDefinitions = program.Types |> List.map (fun declaration -> declaration.Name, declaration.Definition) |> Map.ofList + let resolve = function + | TNamed name as named -> typeDefinitions |> Map.tryFind name |> Option.defaultValue named + | value -> value + let compatible left right = compatible (resolve left) (resolve right) let functionIds = program.Functions |> List.map _.Id let functions = program.Functions |> List.map (fun fn -> fn.Id, fn) |> Map.ofList let invalidImports = program.ImportedRecords |> List.exists (fun record -> System.String.IsNullOrWhiteSpace(record.ModulePath) || System.String.IsNullOrWhiteSpace(record.MemberName)) || (program.ImportedRecords |> List.map _.Name |> List.distinct |> List.length) <> program.ImportedRecords.Length - if invalidImports then + let invalidTypes = + (program.Types |> List.map _.Name |> List.distinct |> List.length) <> program.Types.Length + || (typeIds @ ids |> List.distinct |> List.length) <> typeIds.Length + ids.Length + || program.Types |> List.exists (fun declaration -> + match declaration.Definition with + | TRecord _ -> false + | TUnion(name, _) -> name <> declaration.Name + | _ -> true) + if invalidTypes then + Error [ { Code = "FSL2016"; Stage = Lowering; Message = "LIR type declarations must have unique IDs and names with valid definitions."; Span = None } ] + elif invalidImports then Error [ { Code = "FSL2012"; Stage = Lowering; Message = "LIR imported record linkage must be complete and uniquely named."; Span = None } ] elif ids.Length <> (ids |> Set.ofList |> Set.count) then Error [ { Code = "FSL2001"; Stage = Lowering; Message = "LIR declaration IDs must be unique."; Span = None } ] @@ -400,7 +435,7 @@ module LirValidation = | TTuple types when types.Length = items.Length -> List.iter2 checkPattern types items | _ -> report "FSL2003" "LIR tuple pattern shape is inconsistent." pattern.Span | Lir.RecordPattern fields -> - match pattern.Type with + match resolve pattern.Type with | TRecord types -> fields |> List.iter (fun (name, item) -> match Map.tryFind name types with Some itemType -> checkPattern itemType item | None -> report "FSL2003" $"Unknown LIR record-pattern field '{name}'." item.Span) | _ -> report "FSL2003" "LIR record pattern requires a record." pattern.Span | Lir.SomePattern item -> match pattern.Type with TOption itemType -> checkPattern itemType item | _ -> report "FSL2003" "LIR some pattern requires an option." pattern.Span @@ -411,7 +446,7 @@ module LirValidation = | TMap(keyType, valueType) -> clauses |> List.iter (fun (key, value) -> checkPattern keyType key; checkPattern valueType value); tail |> Option.iter (checkPattern pattern.Type) | _ -> report "FSL2003" "LIR map pattern requires a map." pattern.Span | Lir.UnionPattern(typeName, caseName, payload) -> - match pattern.Type with + match resolve pattern.Type with | TUnion(actual, cases) when actual = typeName -> match Map.tryFind caseName cases, payload with | Some(Some payloadType), Some value -> checkPattern payloadType value @@ -467,10 +502,10 @@ module LirValidation = match expected with Some(operand, result) when not (arguments |> List.forall (fun item -> compatible operand item.Type) && compatible result expression.Type) -> report "FSL2008" "LIR primitive operand or result type is inconsistent." expression.Span | _ -> () | Lir.Tuple items -> items |> List.iter check; match expression.Type with TTuple types when types.Length = items.Length && List.forall2 (fun expected (item: Lir.Expression) -> compatible expected item.Type) types items -> () | _ -> report "FSL2004" "LIR tuple representation is inconsistent." expression.Span | Lir.List items -> items |> List.iter check; match expression.Type with TList itemType when items |> List.forall (fun item -> compatible itemType item.Type) -> () | _ -> report "FSL2004" "LIR list representation is inconsistent." expression.Span - | Lir.Record fields -> fields |> List.iter (snd >> check); match expression.Type with TRecord types when fields.Length = types.Count && fields |> List.forall (fun (name, item) -> Map.tryFind name types |> Option.exists (fun expected -> compatible expected item.Type)) -> () | _ -> report "FSL2004" "LIR record representation is inconsistent." expression.Span + | Lir.Record fields -> fields |> List.iter (snd >> check); match resolve expression.Type with TRecord types when fields.Length = types.Count && fields |> List.forall (fun (name, item) -> Map.tryFind name types |> Option.exists (fun expected -> compatible expected item.Type)) -> () | _ -> report "FSL2004" "LIR record representation is inconsistent." expression.Span | Lir.FieldGet(target, name) -> check target - match target.Type, name with + match resolve target.Type, name with | TRecord fields, _ when Map.tryFind name fields |> Option.exists (compatible expression.Type) -> () | TNamed "Environment", "ScriptName" when compatible expression.Type (TOption TString) -> () | TNamed "Environment", "Arguments" when compatible expression.Type (TList TString) -> () @@ -485,10 +520,10 @@ module LirValidation = | Lir.DecisionTree(target, cases) -> check target; cases |> List.iter (fun case -> let caseLocals = Set.union locals (patternBindings case.Pattern) in checkPattern target.Type case.Pattern; case.Guard |> Option.iter (fun guard -> checkExpression caseLocals guard; if not (compatible guard.Type TBool) then report "FSL2007" "LIR decision guard must be Boolean." guard.Span); checkExpression caseLocals case.Body; if not (compatible expression.Type case.Body.Type) then report "FSL2007" "LIR decision body type is inconsistent." case.Body.Span) | Lir.Range(first, last) -> check first; check last; if not (compatible first.Type TInt && compatible last.Type TInt && compatible expression.Type (TList TInt)) then report "FSL2004" "LIR range representation is inconsistent." expression.Span | Lir.RecordUpdate(target, fields) -> check target; fields |> List.iter (snd >> check); if not (compatible expression.Type target.Type) then report "FSL2004" "LIR record update representation is inconsistent." expression.Span - | Lir.RecordConvert(target, targetType) -> check target; match target.Type, targetType with TRecord _, (TNamed _ | TRecord _) when compatible expression.Type targetType -> () | TNamed _, TRecord _ when compatible expression.Type targetType -> () | _ -> report "FSL2004" "LIR record conversion evidence is inconsistent." expression.Span + | Lir.RecordConvert(target, targetType) -> check target; match resolve target.Type, resolve targetType with TRecord _, TRecord _ when compatible expression.Type targetType -> () | _ -> report "FSL2004" "LIR record conversion evidence is inconsistent." expression.Span | Lir.IndexGet(target, index) -> check target; check index; match target.Type, expression.Type with TList item, TOption result when compatible index.Type TInt && compatible item result -> () | TMap(key, value), TOption result when compatible key index.Type && compatible value result -> () | _ -> report "FSL2004" "LIR index representation is inconsistent." expression.Span | Lir.Interpolate parts -> parts |> List.iter (function Lir.TextPart _ -> () | Lir.ValuePart value -> check value); if not (compatible expression.Type TString) then report "FSL2004" "LIR interpolation must produce string." expression.Span - | Lir.UnionCase(typeName, caseName, payload) -> payload |> Option.iter check; match expression.Type with TUnion(actual, cases) when actual = typeName && Map.containsKey caseName cases -> () | _ -> report "FSL2004" "LIR union representation is inconsistent." expression.Span + | Lir.UnionCase(typeName, caseName, payload) -> payload |> Option.iter check; match resolve expression.Type with TUnion(actual, cases) when actual = typeName && Map.containsKey caseName cases -> () | _ -> report "FSL2004" "LIR union representation is inconsistent." expression.Span | Lir.Map entries -> entries |> List.iter (function Lir.KeyValue(key, value) -> check key; check value | Lir.Spread value -> check value); match expression.Type with TMap _ -> () | _ -> report "FSL2004" "LIR map node requires a map type." expression.Span | Lir.Intrinsic name when not (Map.containsKey name BuiltinSignatures.builtinSchemes || Map.containsKey name RuntimeIntrinsics.schemes) -> report "FSL2009" $"Unknown LIR intrinsic '{name}'." expression.Span | Lir.HostExternal(_, arity) when arity < 0 -> report "FSL2010" "LIR host external arity cannot be negative." expression.Span diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index 4518293..57c0e34 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -76,6 +76,17 @@ type GoBackendTests() = | Ok _ -> Assert.Fail("Expected imported-record diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSB4005")) + [] + member _.``Go backend emits concrete ordinary record types`` () = + let source = "type Person = { Name: string; Age: int }\nlet person = { Name = \"Ada\"; Age = 42 }\nperson.Name" + let compiled = match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with Ok value -> value | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + match GoBackend.generate { ModuleName = "fscript.generated/records" } compiled.Lir with + | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) + | Ok files -> + let program = files |> List.find (fun file -> file.RelativePath.EndsWith("program.go")) |> _.Content + Assert.That(program, Does.Match("type record[0-9]+ struct")) + Assert.That(program, Does.Contain("`fscript:\"Name\"`")) + [] [] [] @@ -122,6 +133,9 @@ type GoBackendTests() = [] [] let upper : string -> string\nupper \"hello\"", "HELLO")>] [] + [] + [] + [] member _.``Generated projects run with Go`` (source: string, expected: string) = requireGo () let input = { Program = Parser.parseProgram source; Externs = [] } diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index 86a746d..db8c724 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -39,13 +39,27 @@ type HirTests() = let declarations = compiled.Lir.Declarations |> List.filter (fun declaration -> declaration.Name = "add" || declaration.Name.StartsWith("$expression")) Assert.That(Lir.format { compiled.Lir with Declarations = declarations }, Is.EqualTo(snapshot "add.lir")) + [] + member _.``HIR and LIR preserve named type declarations`` () = + let source = "type Person = { Name: string; Age: int }\nlet person = { Name = \"Ada\"; Age = 42 }\nperson.Name" + let compiled = + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Ok value -> value + | Error diagnostics -> failwith (System.String.Join("; ", diagnostics |> List.map _.Message)) + let hirType = compiled.Hir.Types |> List.exactlyOne + let lirType = compiled.Lir.Types |> List.exactlyOne + Assert.That(hirType.Name, Is.EqualTo("Person")) + Assert.That((match hirType.Definition with TRecord _ -> true | _ -> false), Is.True) + Assert.That(lirType.Name, Is.EqualTo("Person")) + Assert.That(compiled.Hir.Declarations |> List.exists (fun declaration -> declaration.Value.Type = TNamed "Person"), Is.True) + [] member _.``HIR validation rejects a declaration scheme mismatch`` () = let value: Hir.Expression = { Type = TInt; Span = span; Node = Hir.Constant(Hir.Int 1L) } let declaration: Hir.Declaration = { Id = Hir.SymbolId 0; Name = "value"; Scheme = Forall([], TString) IsRecursive = false; IsExported = false; Export = None; Span = span; Value = value } - match HirValidation.validate { ImportedRecords = []; Declarations = [ declaration ]; Entry = None } with + match HirValidation.validate { Types = []; ImportedRecords = []; Declarations = [ declaration ]; Entry = None } with | Ok _ -> Assert.Fail("Expected invalid HIR diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSH1104")) @@ -55,19 +69,19 @@ type HirTests() = let branch: Lir.Expression = { Type = TInt; Span = span Node = Lir.Branch(constant TInt (Lir.Int 1L), constant TInt (Lir.Int 2L), constant TString (Lir.String "no")) } - match LirValidation.validate { ImportedRecords = []; Functions = []; Declarations = []; Entry = Some branch } with + match LirValidation.validate { Types = []; ImportedRecords = []; Functions = []; Declarations = []; Entry = Some branch } with | Ok _ -> Assert.Fail("Expected invalid LIR diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSL2007")) [] member _.``LIR validation rejects unknown primitives`` () = let intrinsic = ({ Type = TInt; Span = span; Node = Lir.Intrinsic "Unknown.value" }: Lir.Expression) - match LirValidation.validate { ImportedRecords = []; Functions = []; Declarations = []; Entry = Some intrinsic } with + match LirValidation.validate { Types = []; ImportedRecords = []; Functions = []; Declarations = []; Entry = Some intrinsic } with | Ok _ -> Assert.Fail("Expected unavailable primitive diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSL2009")) [] member _.``Initial LIR optimization pass is identity`` () = let expression = ({ Type = TInt; Span = span; Node = Lir.Constant(Lir.Int 1L) }: Lir.Expression) - let program: Lir.Program = { ImportedRecords = []; Functions = []; Declarations = []; Entry = Some expression } + let program: Lir.Program = { Types = []; ImportedRecords = []; Functions = []; Declarations = []; Entry = Some expression } Assert.That(LirOptimization.optimize program = program, Is.True) diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index 2743da9..f9160a2 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -110,6 +110,15 @@ type IrArtifactTests() = | Ok loaded -> Assert.That(loaded, Is.EqualTo(lir)) | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) + [] + member _.``IR artifact preserves named type declarations`` () = + let lir = (compile "type Person = { Name: string; Age: int }\n{ Name = \"Ada\"; Age = 42 }").Lir + let bytes = IrArtifact.serialize lir + Assert.That(Encoding.UTF8.GetString(bytes), Does.Contain("\"types\"")) + match IrArtifact.deserialize bytes with + | Ok loaded -> Assert.That(loaded.Types = lir.Types, Is.True) + | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) + [] member _.``LIR closures serialize deterministic explicit captures`` () = let source = "let outer x = (let inner y = x + y\n inner\n)\nouter 40 2" From b1dc0cf4cf111641aa0391beff2f270a09287374 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 19:50:35 +0200 Subject: [PATCH 045/133] refactor(compiler): preserve resolved symbol identities --- CHANGELOG.md | 2 + docs/architecture/typed-ir-and-backends.md | 2 +- docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Compiler/HirLowering.fs | 55 +++++++++++++++------- tests/FScript.Compiler.Tests/HirTests.fs | 37 +++++++++++++++ 5 files changed, 79 insertions(+), 18 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 077ab51..f9d9930 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Reused resolved frontend symbol IDs unchanged throughout HIR lowering, including interleaved types and local bindings. + - Preserved named type declarations through HIR, LIR, and artifacts and generated concrete Go structs for ordinary records. - Added a canonical LIR debug printer and checked-in deterministic HIR/LIR arithmetic snapshots. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 99d5a31..7b2caa0 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -20,7 +20,7 @@ Validated LIR consumers are separate leaf assemblies. `FScript.Interpreter.Lir` ## IR boundaries -HIR is language-oriented and contains resolved artifact-local symbol identities, ordinary and imported type declarations, complete semantic types and schemes, source locations, structured export metadata, and explicit nodes for named/structural record conversions. Named record and union references remain nominal in HIR and LIR; their declaration shapes are carried separately and serialized in artifacts. Export metadata retains the exported source name and span through LIR and artifacts while the legacy Boolean remains available to compatibility consumers. Conversion evidence is produced by inference, not rediscovered from backend representations. HIR contains no Go types. +HIR is language-oriented and contains resolved artifact-local symbol identities, ordinary and imported type declarations, complete semantic types and schemes, source locations, structured export metadata, and explicit nodes for named/structural record conversions. HIR directly reuses the frontend IDs for builtins, externs, types, constructors, declarations, parameters, pattern bindings, and locals; only compiler-synthesized nodes allocate IDs above the frontend range. Named record and union references remain nominal in HIR and LIR; their declaration shapes are carried separately and serialized in artifacts. Export metadata retains the exported source name and span through LIR and artifacts while the legacy Boolean remains available to compatibility consumers. Conversion evidence is produced by inference, not rediscovered from backend representations. HIR contains no Go types. LIR is executable but not a Go AST. Lambda bodies are lifted into a deterministic program-level function table. Closure creation references a function ID and carries its canonical ordered set of captured symbol IDs, alongside resolved primitive operations, aggregate construction/projection, and structured control flow. Validation recomputes each lifted function's free-symbol set and rejects unknown function IDs or forged, missing, duplicate, or unordered captures. Pattern matching becomes deterministic tests and projections before reaching a backend. Semantic types remain alongside selected runtime representations for validation and diagnostics. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index f0d17f8..6f7b3da 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -10,6 +10,7 @@ This document records the behavior that the typed IR, LIR interpreter, and compi - Hindley-Milner inference generalizes top-level and local let bindings after applying substitutions. Recursive bindings are inferred monomorphically inside their recursive group and generalized afterward. - The active typed program stores top-level types but retains parser expressions in bodies. Compiler lowering therefore requires a new resolved, recursively typed result emitted by the inference pass; types cannot safely be reconstructed afterward. - Resolved inference exposes a recursive expression tree whose nodes carry their final instantiated type, span, lexical reference, resolved primitive operation, and named/structural record-coercion evidence. It also exposes deterministic symbols for every lexical binding category and links each captured reference to its target ID. Type-variable numbering in resolved schemes and reference types is normalized independently of prior inference calls. +- HIR reuses those resolved IDs unchanged for source symbols even when type and value declarations are interleaved; compiler-synthesized temporaries and constructor payload parameters occupy IDs above the resolved frontend range. - Top-level statements execute in source order. Every non-final expression must have type `unit`; the final expression is the program result. - Compiled top-level expressions execute exactly once in source order even when interleaved with declarations. Top-level pattern bindings evaluate their scrutinee once, fail before later statements when unmatched, and expose all bound names to subsequent statements. diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index df656e4..657403b 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -9,6 +9,7 @@ module HirLowering = Coercions: Map mutable UnionCases: Map mutable TypeDefinitions: Map + mutable AvailableSymbols: TypeInfer.ResolvedSymbol list mutable Diagnostics: CompilerDiagnostic list } let private newSymbol state = @@ -16,6 +17,21 @@ module HirLowering = state.NextSymbol <- value + 1 Hir.SymbolId value + let private claimSymbol state kind name span = + match state.AvailableSymbols |> List.tryFind (fun symbol -> symbol.Kind = kind && symbol.Name = name && symbol.Span = span) with + | Some symbol -> + state.AvailableSymbols <- state.AvailableSymbols |> List.filter (fun candidate -> candidate.SymbolId <> symbol.SymbolId) + Hir.SymbolId symbol.SymbolId + | None -> + state.Diagnostics <- state.Diagnostics @ [ Diagnostic.hir "FSH1007" $"Resolved symbol '{name}' was not available during HIR lowering." (Some span) ] + newSymbol state + + let private findSymbol state kind name span = + state.AvailableSymbols + |> List.tryFind (fun symbol -> symbol.Kind = kind && symbol.Name = name && symbol.Span = span) + |> Option.map (fun symbol -> Hir.SymbolId symbol.SymbolId) + |> Option.defaultWith (fun () -> claimSymbol state kind name span) + let rec private representationType state = function | TFun(argument, result) -> TFun(representationType state argument, representationType state result) | TTuple items -> TTuple(List.map (representationType state) items) @@ -82,7 +98,7 @@ module HirLowering = match pattern with | PWildcard _ -> Ok(make Hir.Wildcard, Map.empty) | PVar(name, _) -> - let id = newSymbol state + let id = claimSymbol state TypeInfer.PatternBindingSymbol name span Ok(make (Hir.Binding(id, name)), Map.ofList [ name, id ]) | PLiteral(value, _) -> let lowered, _ = literal value in Ok(make (Hir.LiteralPattern lowered), Map.empty) | PNil _ -> Ok(make Hir.EmptyList, Map.empty) @@ -157,7 +173,7 @@ module HirLowering = | ELambda(parameter, body, _) -> match inferred with | TFun(parameterType, resultType) -> - let id = newSymbol state + let id = claimSymbol state TypeInfer.ParameterSymbol parameter.Name parameter.Span lowerExpression state (environment |> Map.add parameter.Name id) resultType body |> Result.map (fun lowered -> make (Hir.Lambda(id, parameter.Name, parameterType, lowered))) | _ -> Error [ Diagnostic.hir "FSH1002" "Lambda does not have a function type." (Some span) ] @@ -247,7 +263,7 @@ module HirLowering = match lowerExpression state environment inferred left, lowerExpression state environment inferred right with | Ok l, Ok r -> Ok(make (Hir.Append(l, r))) | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e | ELet(name, binding, body, isRecursive, _, _) -> - let id = newSymbol state + let id = claimSymbol state TypeInfer.LocalSymbol name span let bindingType = expressionType state binding (Types.freshVar()) let bodyEnvironment = environment |> Map.add name id let bindingEnvironment = if isRecursive then bodyEnvironment else environment @@ -256,7 +272,7 @@ module HirLowering = | Error a, Error b -> Error(a @ b) | Error errors, _ | _, Error errors -> Error errors | ELetRecGroup(bindings, body, _) -> - let ids = bindings |> List.map (fun (name, _, _, _, _) -> name, newSymbol state) |> Map.ofList + let ids = bindings |> List.map (fun (name, _, _, _, bindingSpan) -> name, claimSymbol state TypeInfer.LocalSymbol name bindingSpan) |> Map.ofList let recursiveEnvironment = Map.fold (fun env name id -> Map.add name id env) environment ids let loweredBindings = bindings @@ -282,7 +298,7 @@ module HirLowering = | EFor(name, source, body, _) -> let sourceType = expressionType state source (TList(Types.freshVar())) let itemType = match sourceType with TList item -> item | _ -> Types.freshVar() - let id = newSymbol state + let id = claimSymbol state TypeInfer.LocalSymbol name span match lowerExpression state environment sourceType source, lowerExpression state (environment |> Map.add name id) TUnit body with | Ok loweredSource, Ok loweredBody -> Ok(make (Hir.ForEach(id, name, itemType, loweredSource, loweredBody))) | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e @@ -356,14 +372,18 @@ module HirLowering = let state = item.RecordCoercion |> Option.map (fun evidence -> Map.add item.Source evidence state) |> Option.defaultValue state item.Children |> List.fold collect state resolved.ResolvedStatements |> List.collect _.Expressions |> List.fold collect Map.empty - let state = { NextSymbol = 0; ExpressionTypes = expressionTypes; Coercions = coercions; UnionCases = Map.empty; TypeDefinitions = Map.empty; Diagnostics = [] } + let nextSynthetic = resolved.Symbols |> List.map _.SymbolId |> function [] -> 0 | values -> List.max values + 1 + let state = + { NextSymbol = nextSynthetic; ExpressionTypes = expressionTypes; Coercions = coercions + UnionCases = Map.empty; TypeDefinitions = Map.empty; AvailableSymbols = resolved.Symbols; Diagnostics = [] } + let syntheticSpan = Span.mk (Span.pos 0 0) (Span.pos 0 0) let builtins = BuiltinSignatures.builtinSchemes |> Map.toList |> List.sortBy fst - |> List.map (fun (name, _) -> name, newSymbol state) + |> List.map (fun (name, _) -> name, findSymbol state TypeInfer.BuiltinSymbol name syntheticSpan) |> Map.ofList - let externSymbols = externs |> List.sortBy _.Name |> List.map (fun value -> value.Name, newSymbol state) |> Map.ofList + let externSymbols = externs |> List.sortBy _.Name |> List.map (fun value -> value.Name, findSymbol state TypeInfer.ExternSymbol value.Name syntheticSpan) |> Map.ofList let typeDefinitions = resolved.TypedProgram |> List.choose (function TypeInfer.TSType definition -> Some definition | _ -> None) let rec typeFromRef = function @@ -393,20 +413,20 @@ module HirLowering = definition.Cases |> List.map (fun (caseName, _) -> caseName, (definition.Name, cases.[caseName]))) |> Map.ofList let typeSymbols = typeDefinitions - |> List.map (fun definition -> definition.Name, newSymbol state) + |> List.map (fun definition -> definition.Name, findSymbol state TypeInfer.TypeSymbol definition.Name definition.Span) |> Map.ofList let constructorSymbols = - unionDefinitions |> List.collect (fun (definition, _) -> definition.Cases |> List.map (fun (caseName, _) -> definition.Name, caseName, newSymbol state)) + unionDefinitions |> List.collect (fun (definition, _) -> definition.Cases |> List.map (fun (caseName, _) -> definition.Name, caseName, findSymbol state TypeInfer.UnionCaseSymbol $"{definition.Name}.{caseName}" definition.Span)) let topLevel = resolved.TypedProgram |> List.collect (function - | TypeInfer.TSLet(name, _, _, _, _, _) -> [ name ] - | TypeInfer.TSLetPattern(_, _, bindings, _, _) -> bindings |> Map.keys |> Seq.toList - | TypeInfer.TSLetRecGroup(bindings, _, _) -> bindings |> List.map (fun (name, _, _, _) -> name) - | TypeInfer.TSBackendImport(name, _, _, _) -> [ name ] + | TypeInfer.TSLet(name, _, _, _, _, span) -> [ name, span ] + | TypeInfer.TSLetPattern(_, _, bindings, _, span) -> bindings |> Map.keys |> Seq.map (fun name -> name, span) |> Seq.toList + | TypeInfer.TSLetRecGroup(bindings, _, _) -> bindings |> List.map (fun (name, _, _, span) -> name, span) + | TypeInfer.TSBackendImport(name, _, _, span) -> [ name, span ] | _ -> []) |> List.distinct - |> List.fold (fun env name -> env |> Map.add name (newSymbol state)) (Map.fold (fun env name id -> Map.add name id env) builtins externSymbols) + |> List.fold (fun env (name, span) -> env |> Map.add name (findSymbol state TypeInfer.DeclarationSymbol name span)) (Map.fold (fun env name id -> Map.add name id env) builtins externSymbols) let topLevel = constructorSymbols |> List.fold (fun env (typeName, caseName, id) -> env |> Map.add caseName id |> Map.add $"{typeName}.{caseName}" id) topLevel @@ -543,5 +563,6 @@ module HirLowering = Definition = state.TypeDefinitions.[definition.Name] Import = importedRecords |> List.tryFind (fun imported -> imported.Name = definition.Name) Span = definition.Span }: Hir.TypeDeclaration)) - if errors.IsEmpty then Ok { Types = hirTypes; ImportedRecords = importedRecords; Declarations = declarations; Entry = entry } - else Error errors + let diagnostics = errors @ state.Diagnostics + if diagnostics.IsEmpty then Ok { Types = hirTypes; ImportedRecords = importedRecords; Declarations = declarations; Entry = entry } + else Error diagnostics diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index db8c724..e14d104 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -53,6 +53,43 @@ type HirTests() = Assert.That(lirType.Name, Is.EqualTo("Person")) Assert.That(compiled.Hir.Declarations |> List.exists (fun declaration -> declaration.Value.Type = TNamed "Person"), Is.True) + [] + member _.``HIR reuses deterministic resolved frontend symbol IDs`` () = + let source = "let first = 1\ntype Choice = | Pick of int\nlet apply x = (let y = x\n match Some y with\n | Some z -> z\n)\napply first" + let program = Parser.parseProgram source + let external name scheme arity = ({ Name = name; Scheme = scheme; Arity = arity; Impl = fun _ _ -> VUnit }: ExternalFunction) + let frontendExterns = + [ external "Env" (Forall([], TNamed "Environment")) 0 + yield! RuntimeIntrinsics.schemes |> Map.toList |> List.map (fun (name, scheme) -> external name scheme 1) + external "Task.spawn" (Forall([ 0 ], TFun(TFun(TUnit, TVar 0), TTask(TVar 0)))) 1 + external "Task.await" (Forall([ 0 ], TFun(TTask(TVar 0), TVar 0))) 1 ] + let resolved = TypeInfer.inferResolvedProgramWithExterns frontendExterns program + let compiled = match Compiler.compile { Program = program; Externs = [] } with Ok value -> value | Error diagnostics -> failwith (System.String.Join("; ", diagnostics |> List.map _.Message)) + let resolvedId kind name = resolved.Symbols |> List.find (fun symbol -> symbol.Kind = kind && symbol.Name = name) |> _.SymbolId + let declarationId name = compiled.Hir.Declarations |> List.find (fun declaration -> declaration.Name = name) |> _.Id |> Hir.symbolNumber + Assert.That(declarationId "first", Is.EqualTo(resolvedId TypeInfer.DeclarationSymbol "first")) + Assert.That(declarationId "Choice.Pick", Is.EqualTo(resolvedId TypeInfer.UnionCaseSymbol "Choice.Pick")) + Assert.That(declarationId "apply", Is.EqualTo(resolvedId TypeInfer.DeclarationSymbol "apply")) + Assert.That(compiled.Hir.Types.Head.Id |> Hir.symbolNumber, Is.EqualTo(resolvedId TypeInfer.TypeSymbol "Choice")) + let rec bindings (expression: Hir.Expression) = + let patternBindings (pattern: Hir.Pattern) = + let rec collect (pattern: Hir.Pattern) = + match pattern.Node with + | Hir.Binding(id, name) -> [ name, Hir.symbolNumber id ] + | Hir.SomePattern item -> collect item + | _ -> [] + collect pattern + match expression.Node with + | Hir.Lambda(id, name, _, body) -> (name, Hir.symbolNumber id) :: bindings body + | Hir.Let(id, name, _, value, body) -> (name, Hir.symbolNumber id) :: (bindings value @ bindings body) + | Hir.Match(target, cases) -> bindings target @ (cases |> List.collect (fun case -> patternBindings case.Pattern @ bindings case.Body)) + | Hir.Apply(left, right) -> bindings left @ bindings right + | _ -> [] + let applyBindings = compiled.Hir.Declarations |> List.find (fun declaration -> declaration.Name = "apply") |> _.Value |> bindings |> Map.ofList + Assert.That(applyBindings.["x"], Is.EqualTo(resolvedId TypeInfer.ParameterSymbol "x")) + Assert.That(applyBindings.["y"], Is.EqualTo(resolvedId TypeInfer.LocalSymbol "y")) + Assert.That(applyBindings.["z"], Is.EqualTo(resolvedId TypeInfer.PatternBindingSymbol "z")) + [] member _.``HIR validation rejects a declaration scheme mismatch`` () = let value: Hir.Expression = { Type = TInt; Span = span; Node = Hir.Constant(Hir.Int 1L) } From e9201da4b72c43ba5f21da67e2dc4d63edc80746 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 19:59:19 +0200 Subject: [PATCH 046/133] feat(go): emit concrete declared unions --- CHANGELOG.md | 2 + docs/architecture/typed-ir-and-backends.md | 2 +- docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Backend.Go/GoBackend.fs | 66 +++++++++++++++++-- src/FScript.Backend.Go/Runtime/patterns.go | 4 +- src/FScript.Backend.Go/Runtime/runtime.go | 44 +++++++++---- .../GoBackendTests.fs | 15 +++++ 7 files changed, 114 insertions(+), 20 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f9d9930..01e973b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Generated distinct tagged Go structs with case-specific payload fields for ordinary declared unions. + - Reused resolved frontend symbol IDs unchanged throughout HIR lowering, including interleaved types and local bindings. - Preserved named type declarations through HIR, LIR, and artifacts and generated concrete Go structs for ordinary records. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 7b2caa0..8473bc0 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -24,7 +24,7 @@ HIR is language-oriented and contains resolved artifact-local symbol identities, LIR is executable but not a Go AST. Lambda bodies are lifted into a deterministic program-level function table. Closure creation references a function ID and carries its canonical ordered set of captured symbol IDs, alongside resolved primitive operations, aggregate construction/projection, and structured control flow. Validation recomputes each lifted function's free-symbol set and rejects unknown function IDs or forged, missing, duplicate, or unordered captures. Pattern matching becomes deterministic tests and projections before reaching a backend. Semantic types remain alongside selected runtime representations for validation and diagnostics. -The Go backend selects concrete generated structs for ordinary named records, with deterministic synthetic field identifiers and FScript-name tags. Structural records remain runtime maps, and explicit LIR conversion nodes copy fields between the representations while evaluating their source once. Runtime record projection, patterns, formatting, and equality reflect only tagged generated fields, so nominal representation does not alter FScript structural behavior. Imported record adapters remain isolated at target-call boundaries. +The Go backend selects concrete generated structs for ordinary named records, with deterministic synthetic field identifiers and FScript-name tags. Declared unions use generated tagged structs with a field for each payload-bearing case and implement a small runtime conversion interface for generic matching and formatting. Structural records remain runtime maps, and explicit LIR conversion nodes copy fields between the representations while evaluating their source once. Runtime record projection, patterns, formatting, and equality reflect only tagged generated fields, so nominal representation does not alter FScript structural behavior. Imported record adapters remain isolated at target-call boundaries. The pass pipeline is HIR lowering, HIR validation, LIR lowering, `LirOptimization.optimize`, then LIR validation. The optimization pass is intentionally identity-only in version 1; its explicit boundary prevents later transformations from being hidden inside lowering or a backend. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 6f7b3da..98ec8ed 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -31,6 +31,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Maps currently accept string keys through inference and have deterministic ordered-map behavior. Runtime support for integer map keys is legacy representation detail, not permission for newly compiled programs. - Declared records retain declaration identity for named annotations. Structural record annotations retain field-shape compatibility and may accept additional fields where current inference permits it. Inference records coercion evidence at the accepting application; HIR and LIR retain an explicit conversion node so backends convert representations without changing compatibility rules. - HIR, LIR, and version-1 artifacts carry ordinary record and union declarations separately from nominal references. Generated Go uses concrete structs for ordinary named records, copies record updates, and performs explicit structural/named conversions; projection, patterns, formatting, and recursive equality remain shape-compatible. +- Generated Go represents each ordinary declared union as its own tagged struct with case-specific payload fields. Runtime union matching, formatting, and recursive equality consume a narrow conversion interface, preserving FScript behavior without erasing declarations to one storage type. - Discriminated unions retain their declared type name, case name, and optional payload. Recursive types require `type rec`; mutually recursive type-definition graphs are rejected. - `typeof` produces an FScript type token and `nameof` produces the resolved source name. - Compiled local pattern bindings evaluate their source once and use the ordinary match-failure path. Compiled `for` evaluates its list once, executes bodies from first to last, discards their values, and returns unit. Compiled `typeof` retains the resolved declared record or union shape in the artifact instead of performing a name lookup at execution time. diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index 41328ce..04b6ed2 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -22,6 +22,9 @@ module GoBackend = let private tryRecordDeclaration name = tryDeclaredType name |> Option.bind (fun declaration -> match declaration.Import, declaration.Definition with None, TRecord fields -> Some(declaration, fields) | _ -> None) + let private tryUnionDeclaration name = + tryDeclaredType name + |> Option.bind (fun declaration -> match declaration.Import, declaration.Definition with None, TUnion(_, cases) -> Some(declaration, cases) | _ -> None) let private recordFieldName fields name = fields |> Map.toList |> List.findIndex (fst >> (=) name) |> sprintf "Field%d" let private effectiveType (expression: Lir.Expression) = @@ -61,6 +64,7 @@ module GoBackend = | TNamed "Environment" -> "map[string]any" | TNamed name when tryImportedRecord name |> Option.isSome -> "map[string]any" | TNamed name when tryRecordDeclaration name |> Option.isSome -> tryRecordDeclaration name |> Option.get |> fst |> declaredTypeName + | TNamed name when tryUnionDeclaration name |> Option.isSome -> tryUnionDeclaration name |> Option.get |> fst |> fun declaration -> $"union{Lir.symbolNumber declaration.Id}" | TUnion _ | TNamed _ -> "fsruntime.Union" | TMap _ -> "fsruntime.OrderedMap" | value -> invalidArg "value" $"Unsupported Go representation for {Types.typeToString value}." @@ -102,7 +106,10 @@ module GoBackend = | TOption _ -> $"fsruntime.AsOption({code})" | TNamed "Environment" -> $"fsruntime.Record({code})" | TNamed name when tryImportedRecord name |> Option.isSome -> $"fsruntime.Record({code})" - | TNamed name when tryRecordDeclaration name |> Option.isSome -> $"{code}.({goType valueType})" + | TNamed name when tryRecordDeclaration name |> Option.isSome -> + let declaration, _ = tryRecordDeclaration name |> Option.get + $"asRecord{Lir.symbolNumber declaration.Id}({code})" + | TNamed name when tryUnionDeclaration name |> Option.isSome -> $"{code}.({goType valueType})" | TUnion _ | TNamed _ -> $"fsruntime.AsUnion({code})" | TMap _ -> $"fsruntime.AsMap({code})" | TFun _ -> $"fsruntime.AsClosure({code})" @@ -426,9 +433,23 @@ module GoBackend = | Lir.UnionCase(typeName, caseName, payload) -> let quotedType = System.Text.Json.JsonSerializer.Serialize(typeName) let quotedCase = System.Text.Json.JsonSerializer.Serialize(caseName) - match payload with - | None -> Ok $"fsruntime.MakeUnion({quotedType}, {quotedCase})" - | Some item -> emitExpression environment item |> Result.map (fun code -> $"fsruntime.MakeUnionValue({quotedType}, {quotedCase}, {code})") + match expression.Type with + | TNamed name -> + match tryUnionDeclaration name with + | Some(declaration, cases) -> + let caseIndex = cases |> Map.toList |> List.findIndex (fst >> (=) caseName) + let unionName = $"union{Lir.symbolNumber declaration.Id}" + match payload with + | None -> Ok $"{unionName}{{Tag: {caseIndex}}}" + | Some item -> emitExpression environment item |> Result.map (fun code -> $"{unionName}{{Tag: {caseIndex}, Case{caseIndex}: {code}}}") + | None -> + match payload with + | None -> Ok $"fsruntime.MakeUnion({quotedType}, {quotedCase})" + | Some item -> emitExpression environment item |> Result.map (fun code -> $"fsruntime.MakeUnionValue({quotedType}, {quotedCase}, {code})") + | _ -> + match payload with + | None -> Ok $"fsruntime.MakeUnion({quotedType}, {quotedCase})" + | Some item -> emitExpression environment item |> Result.map (fun code -> $"fsruntime.MakeUnionValue({quotedType}, {quotedCase}, {code})") | Lir.Map entries -> entries |> List.map (function | Lir.KeyValue(key, value) -> @@ -579,6 +600,41 @@ func Environment() map[string]any { for fieldIndex, (fieldName, fieldType) in fields |> Map.toList |> List.indexed do typeDeclarations.AppendLine($"\tField{fieldIndex} {(goType fieldType).PadRight(typeWidth)} `fscript:{System.Text.Json.JsonSerializer.Serialize(fieldName)}`") |> ignore typeDeclarations.AppendLine("}") |> ignore + typeDeclarations.AppendLine() |> ignore + typeDeclarations.AppendLine($"func asRecord{Lir.symbolNumber declaration.Id}(value any) {declaredTypeName declaration} {{") |> ignore + typeDeclarations.AppendLine("\tfields := fsruntime.Record(value)") |> ignore + let convertedFields = + fields |> Map.toList |> List.mapi (fun index (fieldName, fieldType) -> + let rawField = $"fields[{System.Text.Json.JsonSerializer.Serialize(fieldName)}]" + let converted = cast fieldType rawField + $"Field{index}: {converted}") |> String.concat ", " + typeDeclarations.AppendLine($"\treturn {declaredTypeName declaration}{{{convertedFields}}}") |> ignore + typeDeclarations.AppendLine("}") |> ignore + typeDeclarations.AppendLine() |> ignore + | None, TUnion(typeName, cases) -> + let unionName = $"union{Lir.symbolNumber declaration.Id}" + let caseValues = cases |> Map.toList + let payloads = caseValues |> List.indexed |> List.choose (fun (index, (_, payload)) -> payload |> Option.map (fun payloadType -> index, payloadType)) + let fieldWidth = ("Tag" :: (payloads |> List.map (fun (index, _) -> $"Case{index}"))) |> List.map _.Length |> List.max + typeDeclarations.AppendLine($"type {unionName} struct {{") |> ignore + let tagField = "Tag".PadRight(fieldWidth) + typeDeclarations.AppendLine($"\t{tagField} int") |> ignore + for index, payloadType in payloads do + let payloadField = $"Case{index}".PadRight(fieldWidth) + typeDeclarations.AppendLine($"\t{payloadField} {goType payloadType}") |> ignore + typeDeclarations.AppendLine("}") |> ignore + typeDeclarations.AppendLine() |> ignore + typeDeclarations.AppendLine($"func (value {unionName}) FScriptUnion() fsruntime.Union {{") |> ignore + typeDeclarations.AppendLine("\tswitch value.Tag {") |> ignore + for index, (caseName, payload) in caseValues |> List.indexed do + typeDeclarations.AppendLine($"\tcase {index}:") |> ignore + match payload with + | None -> typeDeclarations.AppendLine($"\t\treturn fsruntime.MakeUnion({System.Text.Json.JsonSerializer.Serialize(typeName)}, {System.Text.Json.JsonSerializer.Serialize(caseName)})") |> ignore + | Some _ -> typeDeclarations.AppendLine($"\t\treturn fsruntime.MakeUnionValue({System.Text.Json.JsonSerializer.Serialize(typeName)}, {System.Text.Json.JsonSerializer.Serialize(caseName)}, value.Case{index})") |> ignore + typeDeclarations.AppendLine("\t}") |> ignore + typeDeclarations.AppendLine("\tpanic(\"Invalid FScript union tag\")") |> ignore + typeDeclarations.AppendLine("}") |> ignore + typeDeclarations.AppendLine() |> ignore | _ -> () let initializers = StringBuilder() let mutable error = None @@ -607,7 +663,7 @@ func Environment() map[string]any { ([ yield! targetImports |> List.map (fun (modulePath, _, _, _) -> modulePath) yield! program.ImportedRecords |> List.map _.ModulePath ] |> List.distinct |> List.sort |> List.map (fun path -> $"\t{importAlias path} {System.Text.Json.JsonSerializer.Serialize(path)}")) |> String.concat "\n" - let typeSource = if typeDeclarations.Length = 0 then "" else typeDeclarations.ToString() + "\n" + let typeSource = typeDeclarations.ToString() let programSource = $"package fscriptprogram\n\nimport (\n{importLines}\n)\n\n{typeSource}{declarations}\nfunc init() {{\n{initializers}}}\n\nfunc Run() any {{ return {entryCode} }}\n" let printExpression = match program.Entry with diff --git a/src/FScript.Backend.Go/Runtime/patterns.go b/src/FScript.Backend.Go/Runtime/patterns.go index bcc52bf..275a366 100644 --- a/src/FScript.Backend.Go/Runtime/patterns.go +++ b/src/FScript.Backend.Go/Runtime/patterns.go @@ -111,7 +111,7 @@ func matchRuntimePattern(value any, pattern Pattern) (map[int]any, bool) { } return bindings, true case "union": - item, ok := value.(Union) + item, ok := TryUnion(value) if !ok || item.TypeName != pattern.TypeName || item.CaseName != pattern.CaseName || (len(pattern.Items) == 0) != !item.HasValue { return nil, false } @@ -213,7 +213,7 @@ func TypeMatches(value any, typeName string) bool { _, ok := TryRecord(value) return ok } - if item, ok := value.(Union); ok { + if item, ok := TryUnion(value); ok { return item.TypeName == typeName } _, record := TryRecord(value) diff --git a/src/FScript.Backend.Go/Runtime/runtime.go b/src/FScript.Backend.Go/Runtime/runtime.go index a535676..b078749 100644 --- a/src/FScript.Backend.Go/Runtime/runtime.go +++ b/src/FScript.Backend.Go/Runtime/runtime.go @@ -19,6 +19,9 @@ type Union struct { HasValue bool Value any } +type UnionCarrier interface { + FScriptUnion() Union +} type MapEntry struct { Key any Value any @@ -68,8 +71,23 @@ func Record(value any) map[string]any { } return record } -func AsOption(value any) Option { return value.(Option) } -func AsUnion(value any) Union { return value.(Union) } +func AsOption(value any) Option { return value.(Option) } +func TryUnion(value any) (Union, bool) { + if item, ok := value.(Union); ok { + return item, true + } + if item, ok := value.(UnionCarrier); ok { + return item.FScriptUnion(), true + } + return Union{}, false +} +func AsUnion(value any) Union { + item, ok := TryUnion(value) + if !ok { + panic("Union access requires a union") + } + return item +} func AsMap(value any) OrderedMap { return value.(OrderedMap) } func Some(value any) Option { return Option{HasValue: true, Value: value} } func None() Option { return Option{} } @@ -96,6 +114,12 @@ func HasField(value any, name string) bool { return found } func normalizeStructural(value any) any { + if item, ok := TryUnion(value); ok { + if item.HasValue { + item.Value = normalizeStructural(item.Value) + } + return item + } reflected := reflect.ValueOf(value) if reflected.IsValid() && (reflected.Kind() == reflect.Struct || (reflected.Kind() == reflect.Pointer && !reflected.IsNil() && reflected.Elem().Kind() == reflect.Struct)) { if record, ok := TryRecord(value); ok { @@ -130,11 +154,6 @@ func normalizeStructural(value any) any { item.Value = normalizeStructural(item.Value) } return item - case Union: - if item.HasValue { - item.Value = normalizeStructural(item.Value) - } - return item case OrderedMap: entries := make([]MapEntry, len(item.Entries)) for index, entry := range item.Entries { @@ -333,6 +352,12 @@ func Intrinsic(name string, args []any) any { } } func Format(value any) string { + if item, ok := TryUnion(value); ok { + if !item.HasValue { + return item.CaseName + } + return item.CaseName + " " + Format(item.Value) + } switch item := value.(type) { case bool: if item { @@ -344,11 +369,6 @@ func Format(value any) string { return "None" } return "Some " + Format(item.Value) - case Union: - if !item.HasValue { - return item.CaseName - } - return item.CaseName + " " + Format(item.Value) case OrderedMap: parts := make([]string, len(item.Entries)) for index, entry := range item.Entries { diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index 57c0e34..871e482 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -87,6 +87,19 @@ type GoBackendTests() = Assert.That(program, Does.Match("type record[0-9]+ struct")) Assert.That(program, Does.Contain("`fscript:\"Name\"`")) + [] + member _.``Go backend emits tagged ordinary union types`` () = + let source = "type Shape = | Point | Circle of int\nCircle 3" + let compiled = match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with Ok value -> value | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + match GoBackend.generate { ModuleName = "fscript.generated/unions" } compiled.Lir with + | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) + | Ok files -> + let program = files |> List.find (fun file -> file.RelativePath.EndsWith("program.go")) |> _.Content + Assert.That(program, Does.Match("type union[0-9]+ struct")) + Assert.That(program, Does.Contain("Tag")) + Assert.That(program, Does.Contain("Case0 int64")) + Assert.That(program, Does.Contain("FScriptUnion() fsruntime.Union")) + [] [] [] @@ -136,6 +149,8 @@ type GoBackendTests() = [] [] [] + [] + [] member _.``Generated projects run with Go`` (source: string, expected: string) = requireGo () let input = { Program = Parser.parseProgram source; Externs = [] } From ad1a8d2a4f3afaa7ffbdb74bb10390ae766a4060 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 20:05:43 +0200 Subject: [PATCH 047/133] test(go): broaden structural conformance --- CHANGELOG.md | 2 ++ docs/architecture/typed-ir-and-backends.md | 2 ++ docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Backend.Go/Runtime/runtime.go | 22 ++++++++++------ src/FScript.Compiler/HirLowering.fs | 17 ++++++++----- tests/FScript.Compiler.Tests/HirTests.fs | 7 ++++++ .../FScript.Compiler.Tests/Snapshots/add.hir | 2 +- .../ConformanceTests.fs | 25 +++++++++++++------ 8 files changed, 55 insertions(+), 23 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 01e973b..65788f2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Consumed normalized frontend declaration schemes in HIR and expanded cross-runtime aggregate, closure, record, union, and map conformance. + - Generated distinct tagged Go structs with case-specific payload fields for ordinary declared unions. - Reused resolved frontend symbol IDs unchanged throughout HIR lowering, including interleaved types and local bindings. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 8473bc0..5607876 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -30,6 +30,8 @@ The pass pipeline is HIR lowering, HIR validation, LIR lowering, `LirOptimizatio HIR and LIR expose canonical debug printers with stable artifact-local IDs and normalized type rendering. The arithmetic vertical slice is checked in as paired snapshots so changes to symbol allocation, function lifting, captures, or operation selection are reviewed explicitly. +HIR declaration schemes are taken directly from the resolved frontend after deterministic type-variable normalization; lowering does not generalize source declarations again. Synthetic declarations may derive a local scheme because they have no frontend declaration counterpart. + HIR validation rejects duplicate and unresolved symbols, invalid declaration spans, and schemes inconsistent with their values. LIR validation rechecks artifact-local references and spans, then verifies closure calls, branches and decision guards, primitive signatures, aggregate representations, declaration value types, intrinsic catalog membership, and complete target-import linkage. Artifact deserialization runs the same LIR validator, so malformed JSON cannot bypass compiler invariants. Lexical `let` is retained as a structured LIR node. Its symbol is visible only in the body, or in both the binding and body for recursive functions. The interpreter closes the recursive environment explicitly; Go emission declares the function slot before assigning its closure. `raise` becomes an explicit `Fail` node so consumers preserve evaluation order and attach their own stage-specific failure diagnostic. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 98ec8ed..2ab4a83 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -45,6 +45,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Record updates evaluate the base once and then updates in source order. Ranges, loops, interpolation, indexing, raise, append, and cons preserve the evaluator's existing evaluation order and failures. - Equality is structural for primitives, tuples, records, maps, options, lists, unions, and type tokens, recursively. Functions, tasks, constructors, and extern handles are not structurally equal. Backend host-language comparability must not leak into this behavior. - Cross-runtime tests compare the legacy evaluator with artifact-round-tripped LIR execution and generated Go for successful values and failure categories. Named records passed through typed function boundaries keep record field projection semantics; their nominal annotation must not select a union runtime representation. +- Cross-runtime aggregate formatting quotes nested strings while leaving top-level strings and ordered-map string keys unquoted; lists use compact semicolon separators and tuples retain comma-space separators. - HIR and LIR are accepted only when artifact-local references resolve, source spans are valid, declaration and expression types agree, calls and control-flow nodes have consistent signatures, aggregate representations match their semantic types, and referenced primitives/imports are available and complete. - Task signatures and host linkage remain representable in compiled artifacts. An unused task function handle is inert; fully applying `Task.spawn` or `Task.await` in LIR reports `FSI3021`, while a Go program whose reachable declaration graph requires a task reports `FSB4004`. - Compiler extern input is declarative: it carries a name, type scheme, arity, and linkage classification, never an executable `.NET` delegate. Host execution values remain confined to the legacy/runtime API. diff --git a/src/FScript.Backend.Go/Runtime/runtime.go b/src/FScript.Backend.Go/Runtime/runtime.go index b078749..612d577 100644 --- a/src/FScript.Backend.Go/Runtime/runtime.go +++ b/src/FScript.Backend.Go/Runtime/runtime.go @@ -351,12 +351,13 @@ func Intrinsic(name string, args []any) any { panic("Unsupported FScript intrinsic: " + name) } } -func Format(value any) string { +func Format(value any) string { return formatValue(value, false) } +func formatValue(value any, nested bool) string { if item, ok := TryUnion(value); ok { if !item.HasValue { return item.CaseName } - return item.CaseName + " " + Format(item.Value) + return item.CaseName + " " + formatValue(item.Value, true) } switch item := value.(type) { case bool: @@ -364,27 +365,32 @@ func Format(value any) string { return "true" } return "false" + case string: + if nested { + return strconv.Quote(item) + } + return item case Option: if !item.HasValue { return "None" } - return "Some " + Format(item.Value) + return "Some " + formatValue(item.Value, true) case OrderedMap: parts := make([]string, len(item.Entries)) for index, entry := range item.Entries { - parts[index] = "[" + Format(entry.Key) + "] = " + Format(entry.Value) + parts[index] = "[" + formatValue(entry.Key, false) + "] = " + formatValue(entry.Value, true) } return "{ " + strings.Join(parts, "; ") + " }" case []any: parts := make([]string, len(item)) for index, value := range item { - parts[index] = Format(value) + parts[index] = formatValue(value, true) } return "[" + strings.Join(parts, ";") + "]" case Tuple: parts := make([]string, len(item)) for index, value := range item { - parts[index] = Format(value) + parts[index] = formatValue(value, true) } return "(" + strings.Join(parts, ", ") + ")" case map[string]any: @@ -395,13 +401,13 @@ func Format(value any) string { sort.Strings(names) parts := make([]string, len(names)) for index, name := range names { - parts[index] = name + " = " + Format(item[name]) + parts[index] = name + " = " + formatValue(item[name], true) } return "{ " + strings.Join(parts, "; ") + " }" default: if reflected := reflect.ValueOf(value); reflected.IsValid() && (reflected.Kind() == reflect.Struct || (reflected.Kind() == reflect.Pointer && !reflected.IsNil() && reflected.Elem().Kind() == reflect.Struct)) { if record, ok := TryRecord(value); ok { - return Format(record) + return formatValue(record, nested) } } return fmt.Sprint(value) diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index 657403b..6106ce1 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -376,6 +376,11 @@ module HirLowering = let state = { NextSymbol = nextSynthetic; ExpressionTypes = expressionTypes; Coercions = coercions UnionCases = Map.empty; TypeDefinitions = Map.empty; AvailableSymbols = resolved.Symbols; Diagnostics = [] } + let resolvedSchemes = + resolved.Symbols + |> List.choose (fun symbol -> symbol.Scheme |> Option.map (fun scheme -> Hir.SymbolId symbol.SymbolId, scheme)) + |> Map.ofList + let schemeFor id fallback = resolvedSchemes |> Map.tryFind id |> Option.defaultValue fallback let syntheticSpan = Span.mk (Span.pos 0 0) (Span.pos 0 0) let builtins = BuiltinSignatures.builtinSchemes @@ -435,7 +440,7 @@ module HirLowering = BuiltinSignatures.builtinSchemes |> Map.toList |> List.sortBy fst |> List.map (fun (name, scheme) -> let (Forall(_, valueType)) = scheme let span = Span.mk (Span.pos 0 0) (Span.pos 0 0) - ({ Id = builtins.[name]; Name = name; Scheme = scheme; IsRecursive = false; IsExported = false; Export = None; Span = span; Value = { Type = valueType; Span = span; Node = Hir.Intrinsic name } }: Hir.Declaration)) + ({ Id = builtins.[name]; Name = name; Scheme = schemeFor builtins.[name] scheme; IsRecursive = false; IsExported = false; Export = None; Span = span; Value = { Type = valueType; Span = span; Node = Hir.Intrinsic name } }: Hir.Declaration)) let externalDeclarations = externs |> List.sortBy _.Name |> List.map (fun external -> let (Forall(_, valueType)) = external.Scheme @@ -445,7 +450,7 @@ module HirLowering = | RuntimeContextIntrinsic -> Hir.RuntimeContext | PortableRuntimeIntrinsic -> Hir.Intrinsic external.Name | UnsupportedHostExtern -> Hir.HostExternal(external.Name, external.Arity) - ({ Id = externSymbols.[external.Name]; Name = external.Name; Scheme = external.Scheme; IsRecursive = false; IsExported = false; Export = None; Span = span; Value = { Type = valueType; Span = span; Node = node } }: Hir.Declaration)) + ({ Id = externSymbols.[external.Name]; Name = external.Name; Scheme = schemeFor externSymbols.[external.Name] external.Scheme; IsRecursive = false; IsExported = false; Export = None; Span = span; Value = { Type = valueType; Span = span; Node = node } }: Hir.Declaration)) builtinDeclarations @ externalDeclarations @ (constructorSymbols |> List.map (fun (typeName, caseName, id) -> let definition, _ = unionDefinitions |> List.find (fun (definition, _) -> definition.Name = typeName) let unionType = TNamed typeName @@ -468,7 +473,7 @@ module HirLowering = declarations <- declarations @ [ ({ Id = Map.find name topLevel; Name = name; - Scheme = Types.generalize Map.empty inferred; + Scheme = schemeFor (Map.find name topLevel) (Types.generalize Map.empty inferred); IsRecursive = isRecursive; IsExported = isExported; Export = (exportMetadata name span isExported); @@ -518,7 +523,7 @@ module HirLowering = let body = ({ Type = inferred; Span = span; Node = Hir.Reference patternId }: Hir.Expression) let projection = ({ Type = inferred; Span = span; Node = Hir.Match(target, [ { Pattern = loweredPattern; Guard = None; Body = body; Span = span } ]) }: Hir.Expression) declarations <- declarations @ - [ ({ Id = Map.find name topLevel; Name = name; Scheme = Types.generalize Map.empty inferred; IsRecursive = false; IsExported = isExported; Export = (exportMetadata name span isExported); Span = span; Value = projection }: Hir.Declaration) ] + [ ({ Id = Map.find name topLevel; Name = name; Scheme = schemeFor (Map.find name topLevel) (Types.generalize Map.empty inferred); IsRecursive = false; IsExported = isExported; Export = (exportMetadata name span isExported); Span = span; Value = projection }: Hir.Declaration) ] | Error a, Error b -> errors <- errors @ a @ b | Error diagnostics, _ | _, Error diagnostics -> errors <- errors @ diagnostics | TypeInfer.TSLetRecGroup(bindings, isExported, groupSpan) -> @@ -528,7 +533,7 @@ module HirLowering = declarations <- declarations @ [ ({ Id = Map.find name topLevel Name = name - Scheme = Types.generalize Map.empty inferred + Scheme = schemeFor (Map.find name topLevel) (Types.generalize Map.empty inferred) IsRecursive = true IsExported = isExported Export = (exportMetadata name span isExported) @@ -539,7 +544,7 @@ module HirLowering = declarations <- declarations @ [ ({ Id = Map.find name topLevel Name = name - Scheme = Forall([], importedType) + Scheme = schemeFor (Map.find name topLevel) (Forall([], importedType)) IsRecursive = false IsExported = false Export = None diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index e14d104..1dd3d6e 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -90,6 +90,13 @@ type HirTests() = Assert.That(applyBindings.["y"], Is.EqualTo(resolvedId TypeInfer.LocalSymbol "y")) Assert.That(applyBindings.["z"], Is.EqualTo(resolvedId TypeInfer.PatternBindingSymbol "z")) + [] + member _.``HIR consumes normalized resolved declaration schemes`` () = + TypeInfer.inferProgram (Parser.parseProgram "let perturb left right = (left, right)") |> ignore + let compiled = match Compiler.compile { Program = Parser.parseProgram "let identity value = value\nidentity 42"; Externs = [] } with Ok value -> value | Error diagnostics -> failwith (System.String.Join("; ", diagnostics |> List.map _.Message)) + let identity = compiled.Hir.Declarations |> List.find (fun declaration -> declaration.Name = "identity") + Assert.That(identity.Scheme, Is.EqualTo(Forall([ 0 ], TFun(TVar 0, TVar 0)))) + [] member _.``HIR validation rejects a declaration scheme mismatch`` () = let value: Hir.Expression = { Type = TInt; Span = span; Node = Hir.Constant(Hir.Int 1L) } diff --git a/tests/FScript.Compiler.Tests/Snapshots/add.hir b/tests/FScript.Compiler.Tests/Snapshots/add.hir index 13fe6ae..4ef0de8 100644 --- a/tests/FScript.Compiler.Tests/Snapshots/add.hir +++ b/tests/FScript.Compiler.Tests/Snapshots/add.hir @@ -1,4 +1,4 @@ -decl %54 add: ('b -> ('b -> int)) +decl %54 add: ('a -> ('a -> int)) lambda %55 x: 'a : ('a -> ('a -> int)) lambda %56 y: 'a : ('a -> int) primitive IntAdd : int diff --git a/tests/FScript.Conformance.Tests/ConformanceTests.fs b/tests/FScript.Conformance.Tests/ConformanceTests.fs index fc1726a..fa261f6 100644 --- a/tests/FScript.Conformance.Tests/ConformanceTests.fs +++ b/tests/FScript.Conformance.Tests/ConformanceTests.fs @@ -10,23 +10,24 @@ open NUnit.Framework module private Harness = type Outcome = { Value: string; Stdout: string list } - let rec formatLir = function + let rec private formatLirValue nested = function | LirValue.VUnit -> "()" | LirValue.VInt value -> string value | LirValue.VFloat value -> value.ToString("R", Globalization.CultureInfo.InvariantCulture) | LirValue.VBool value -> string value |> _.ToLowerInvariant() - | LirValue.VString value -> value + | LirValue.VString value -> if nested then System.Text.Json.JsonSerializer.Serialize(value) else value | LirValue.VOption None -> "None" - | LirValue.VOption(Some value) -> "Some " + formatLir value - | LirValue.VList values -> "[" + (values |> List.map formatLir |> String.concat "; ") + "]" - | LirValue.VTuple values -> "(" + (values |> List.map formatLir |> String.concat ", ") + ")" - | LirValue.VRecord fields -> "{ " + (fields |> Map.toList |> List.map (fun (name, value) -> name + " = " + formatLir value) |> String.concat "; ") + " }" + | LirValue.VOption(Some value) -> "Some " + formatLirValue true value + | LirValue.VList values -> "[" + (values |> List.map (formatLirValue true) |> String.concat ";") + "]" + | LirValue.VTuple values -> "(" + (values |> List.map (formatLirValue true) |> String.concat ", ") + ")" + | LirValue.VRecord fields -> "{ " + (fields |> Map.toList |> List.map (fun (name, value) -> name + " = " + formatLirValue true value) |> String.concat "; ") + " }" | LirValue.VUnion(_, caseName, None) -> caseName - | LirValue.VUnion(_, caseName, Some value) -> caseName + " " + formatLir value - | LirValue.VMap entries -> "{ " + (entries |> List.map (fun (key, value) -> "[" + formatLir key + "] = " + formatLir value) |> String.concat "; ") + " }" + | LirValue.VUnion(_, caseName, Some value) -> caseName + " " + formatLirValue true value + | LirValue.VMap entries -> "{ " + (entries |> List.map (fun (key, value) -> "[" + formatLirValue false key + "] = " + formatLirValue true value) |> String.concat "; ") + " }" | LirValue.VTypeToken value -> "" | LirValue.VClosure _ | LirValue.VIntrinsic _ | LirValue.VExternal _ | LirValue.VTargetImport _ -> "" | LirValue.VRuntimeContext -> "" + let formatLir value = formatLirValue false value let compile source = match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with @@ -103,6 +104,14 @@ type ConformanceTests() = [] [ value | None -> 0")>] [] + [] + [] + [] + [] + [] + [ value + amount\napplyTwice (add 2) 3")>] + [] + [ head | _ -> 0")>] member _.``Legacy LIR artifact and Go results conform`` (source: string) = let expected = Harness.legacy source match Harness.lir source with From ecf6df88754334cf9651d6234b17bc2ffc72b36c Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 20:11:30 +0200 Subject: [PATCH 048/133] feat(language): retain structured declaration attributes --- CHANGELOG.md | 2 ++ docs/architecture/typed-ir-and-backends.md | 2 ++ docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Compiler/Hir.fs | 13 ++++++++-- src/FScript.Compiler/HirLowering.fs | 26 +++++++++++++++---- src/FScript.Compiler/HirValidation.fs | 10 ++++++- src/FScript.Language/Parser.fs | 2 +- src/FScript.Language/TypeInfer.fs | 14 +++++++++- tests/FScript.Compiler.Tests/HirTests.fs | 10 ++++++- tests/FScript.Language.Tests/ParserTests.fs | 7 ++--- .../ResolvedInferenceTests.fs | 12 ++++++++- 11 files changed, 84 insertions(+), 15 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 65788f2..60eb0f7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Attached normalized structured Export and Import attributes to resolved statements and HIR declarations. + - Consumed normalized frontend declaration schemes in HIR and expanded cross-runtime aggregate, closure, record, union, and map conformance. - Generated distinct tagged Go structs with case-specific payload fields for ordinary declared unions. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 5607876..352ffd4 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -32,6 +32,8 @@ HIR and LIR expose canonical debug printers with stable artifact-local IDs and n HIR declaration schemes are taken directly from the resolved frontend after deterministic type-variable normalization; lowering does not generalize source declarations again. Synthetic declarations may derive a local scheme because they have no frontend declaration counterpart. +Resolved statements expose normalized structured attributes as name, argument list, and source span. HIR attaches these attributes to ordinary value and type declarations while retaining dedicated export and target-import linkage for consumers that need semantic access without reparsing attribute arguments. + HIR validation rejects duplicate and unresolved symbols, invalid declaration spans, and schemes inconsistent with their values. LIR validation rechecks artifact-local references and spans, then verifies closure calls, branches and decision guards, primitive signatures, aggregate representations, declaration value types, intrinsic catalog membership, and complete target-import linkage. Artifact deserialization runs the same LIR validator, so malformed JSON cannot bypass compiler invariants. Lexical `let` is retained as a structured LIR node. Its symbol is visible only in the body, or in both the binding and body for recursive functions. The interpreter closes the recursive environment explicitly; Go emission declares the function slot before assigning its closure. `raise` becomes an explicit `Fail` node so consumers preserve evaluation order and attach their own stage-specific failure diagnostic. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 2ab4a83..47c8f8a 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -63,6 +63,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Compiled `Env` is a runtime-context intrinsic. The LIR interpreter uses its explicit script name and arguments; generated Go uses the executable base name and `os.Args[1:]`. - `Task.spawn` and `Task.await` are host externs with tracked-task semantics, background error propagation, and an error for unawaited tasks. HIR may represent them before LIR interpreters or backends support them. - `[]` is declaration metadata used by embedding hosts and does not change inference. Backend `Import` metadata is separate and will not weaken the declared FScript type. +- Both `[]` and `[]` normalize to a structured `Export` attribute in resolved statements and HIR. `Import` attributes retain their two arguments and attribute span on ordinary HIR function/type declarations in addition to dedicated backend linkage. ## Diagnostics diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs index 1eb3a8a..b056b87 100644 --- a/src/FScript.Compiler/Hir.fs +++ b/src/FScript.Compiler/Hir.fs @@ -105,6 +105,7 @@ module Hir = { Id: SymbolId Name: string Scheme: Scheme + Attributes: AttributeMetadata list IsRecursive: bool IsExported: bool Export: ExportMetadata option @@ -115,6 +116,11 @@ module Hir = { Name: string Span: Span } + and AttributeMetadata = + { Name: string + Arguments: string list + Span: Span } + type ImportedRecord = { Name: string ModulePath: string @@ -128,6 +134,7 @@ module Hir = Name: string IsRecursive: bool Definition: Type + Attributes: AttributeMetadata list Import: ImportedRecord option Span: Span } @@ -222,10 +229,12 @@ module Hir = | TypeToken tokenType -> lines.Add(typed $"typeof {formatType tokenType}") for typeDeclaration in program.Types do - lines.Add(sprintf "type %%%d %s%s = %s" (symbolNumber typeDeclaration.Id) typeDeclaration.Name (if typeDeclaration.IsRecursive then " rec" else "") (formatType typeDeclaration.Definition)) + let attributes = typeDeclaration.Attributes |> List.map _.Name |> function [] -> "" | values -> " attrs=" + String.concat "," values + lines.Add(sprintf "type %%%d %s%s%s = %s" (symbolNumber typeDeclaration.Id) typeDeclaration.Name (if typeDeclaration.IsRecursive then " rec" else "") attributes (formatType typeDeclaration.Definition)) for declaration in program.Declarations do let export = declaration.Export |> Option.map (fun metadata -> $" export={metadata.Name}") |> Option.defaultValue "" - lines.Add(sprintf "decl %%%d %s%s: %s" (symbolNumber declaration.Id) declaration.Name export (formatType (let (Forall(_, value)) = declaration.Scheme in value))) + let attributes = declaration.Attributes |> List.map _.Name |> function [] -> "" | values -> " attrs=" + String.concat "," values + lines.Add(sprintf "decl %%%d %s%s%s: %s" (symbolNumber declaration.Id) declaration.Name export attributes (formatType (let (Forall(_, value)) = declaration.Scheme in value))) expression 2 declaration.Value match program.Entry with | Some entry -> diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index 6106ce1..2253a12 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -361,6 +361,12 @@ module HirLowering = let lower (externs: CompilerExtern list) (resolved: TypeInfer.ResolvedProgram) : Result = let exportMetadata name span enabled = if enabled then Some({ Name = name; Span = span }: Hir.ExportMetadata) else None + let attributesFor statement = + resolved.ResolvedStatements + |> List.tryFind (fun resolvedStatement -> resolvedStatement.Source = statement) + |> Option.map _.Attributes + |> Option.defaultValue [] + |> List.map (fun attribute -> ({ Name = attribute.Name; Arguments = attribute.Arguments; Span = attribute.Span }: Hir.AttributeMetadata)) let expressionTypes = let rec addExpression state (item: TypeInfer.ResolvedExpression) = item.Children |> List.fold addExpression (Map.add item.Source item.Type state) @@ -440,7 +446,7 @@ module HirLowering = BuiltinSignatures.builtinSchemes |> Map.toList |> List.sortBy fst |> List.map (fun (name, scheme) -> let (Forall(_, valueType)) = scheme let span = Span.mk (Span.pos 0 0) (Span.pos 0 0) - ({ Id = builtins.[name]; Name = name; Scheme = schemeFor builtins.[name] scheme; IsRecursive = false; IsExported = false; Export = None; Span = span; Value = { Type = valueType; Span = span; Node = Hir.Intrinsic name } }: Hir.Declaration)) + ({ Id = builtins.[name]; Name = name; Scheme = schemeFor builtins.[name] scheme; Attributes = []; IsRecursive = false; IsExported = false; Export = None; Span = span; Value = { Type = valueType; Span = span; Node = Hir.Intrinsic name } }: Hir.Declaration)) let externalDeclarations = externs |> List.sortBy _.Name |> List.map (fun external -> let (Forall(_, valueType)) = external.Scheme @@ -450,7 +456,7 @@ module HirLowering = | RuntimeContextIntrinsic -> Hir.RuntimeContext | PortableRuntimeIntrinsic -> Hir.Intrinsic external.Name | UnsupportedHostExtern -> Hir.HostExternal(external.Name, external.Arity) - ({ Id = externSymbols.[external.Name]; Name = external.Name; Scheme = schemeFor externSymbols.[external.Name] external.Scheme; IsRecursive = false; IsExported = false; Export = None; Span = span; Value = { Type = valueType; Span = span; Node = node } }: Hir.Declaration)) + ({ Id = externSymbols.[external.Name]; Name = external.Name; Scheme = schemeFor externSymbols.[external.Name] external.Scheme; Attributes = []; IsRecursive = false; IsExported = false; Export = None; Span = span; Value = { Type = valueType; Span = span; Node = node } }: Hir.Declaration)) builtinDeclarations @ externalDeclarations @ (constructorSymbols |> List.map (fun (typeName, caseName, id) -> let definition, _ = unionDefinitions |> List.find (fun (definition, _) -> definition.Name = typeName) let unionType = TNamed typeName @@ -462,18 +468,20 @@ module HirLowering = let parameter = newSymbol state let body = ({ Type = unionType; Span = definition.Span; Node = Hir.UnionCase(typeName, caseName, Some({ Type = itemType; Span = definition.Span; Node = Hir.Reference parameter })) }: Hir.Expression) ({ Type = TFun(itemType, unionType); Span = definition.Span; Node = Hir.Lambda(parameter, "value", itemType, body) }: Hir.Expression) - ({ Id = id; Name = $"{typeName}.{caseName}"; Scheme = Types.generalize Map.empty value.Type; IsRecursive = false; IsExported = false; Export = None; Span = definition.Span; Value = value }: Hir.Declaration))) + ({ Id = id; Name = $"{typeName}.{caseName}"; Scheme = Types.generalize Map.empty value.Type; Attributes = []; IsRecursive = false; IsExported = false; Export = None; Span = definition.Span; Value = value }: Hir.Declaration))) let mutable entry: Hir.Expression option = None let mutable errors = [] for statement in resolved.TypedProgram do match statement with | TypeInfer.TSLet(name, expression, inferred, isRecursive, isExported, span) -> + let statement = TypeInfer.TSLet(name, expression, inferred, isRecursive, isExported, span) match lowerExpression state topLevel inferred expression with | Ok value -> declarations <- declarations @ [ ({ Id = Map.find name topLevel; Name = name; Scheme = schemeFor (Map.find name topLevel) (Types.generalize Map.empty inferred); + Attributes = attributesFor statement; IsRecursive = isRecursive; IsExported = isExported; Export = (exportMetadata name span isExported); @@ -489,6 +497,7 @@ module HirLowering = [ ({ Id = id Name = name Scheme = Types.generalize Map.empty expression.Type + Attributes = [] IsRecursive = false IsExported = false Export = None @@ -497,6 +506,7 @@ module HirLowering = entry <- Some { Type = expression.Type; Span = value.Span; Node = Hir.Reference id } | Error diagnostics -> errors <- errors @ diagnostics | TypeInfer.TSLetPattern(pattern, expression, bindings, isExported, span) -> + let statement = TypeInfer.TSLetPattern(pattern, expression, bindings, isExported, span) let bindingType = expressionType state expression (Types.freshVar()) match lowerExpression state topLevel bindingType expression, lowerPattern state bindingType pattern with | Ok value, Ok(loweredPattern, patternBindings) -> @@ -505,6 +515,7 @@ module HirLowering = [ ({ Id = temporaryId Name = $"$pattern{Hir.symbolNumber temporaryId}" Scheme = Types.generalize Map.empty bindingType + Attributes = [] IsRecursive = false IsExported = false Export = None @@ -515,7 +526,7 @@ module HirLowering = let check = ({ Type = TUnit; Span = span; Node = Hir.Match(target, [ { Pattern = loweredPattern; Guard = None; Body = unitBody; Span = span } ]) }: Hir.Expression) let checkId = newSymbol state declarations <- declarations @ - [ ({ Id = checkId; Name = $"$patternCheck{Hir.symbolNumber checkId}"; Scheme = Forall([], TUnit); IsRecursive = false; IsExported = false; Export = None; Span = span; Value = check }: Hir.Declaration) ] + [ ({ Id = checkId; Name = $"$patternCheck{Hir.symbolNumber checkId}"; Scheme = Forall([], TUnit); Attributes = []; IsRecursive = false; IsExported = false; Export = None; Span = span; Value = check }: Hir.Declaration) ] for name, inferred in bindings |> Map.toList do match patternBindings |> Map.tryFind name with | None -> errors <- errors @ [ Diagnostic.hir "FSH1006" $"Pattern binding '{name}' was not resolved." (Some span) ] @@ -523,10 +534,11 @@ module HirLowering = let body = ({ Type = inferred; Span = span; Node = Hir.Reference patternId }: Hir.Expression) let projection = ({ Type = inferred; Span = span; Node = Hir.Match(target, [ { Pattern = loweredPattern; Guard = None; Body = body; Span = span } ]) }: Hir.Expression) declarations <- declarations @ - [ ({ Id = Map.find name topLevel; Name = name; Scheme = schemeFor (Map.find name topLevel) (Types.generalize Map.empty inferred); IsRecursive = false; IsExported = isExported; Export = (exportMetadata name span isExported); Span = span; Value = projection }: Hir.Declaration) ] + [ ({ Id = Map.find name topLevel; Name = name; Scheme = schemeFor (Map.find name topLevel) (Types.generalize Map.empty inferred); Attributes = attributesFor statement; IsRecursive = false; IsExported = isExported; Export = (exportMetadata name span isExported); Span = span; Value = projection }: Hir.Declaration) ] | Error a, Error b -> errors <- errors @ a @ b | Error diagnostics, _ | _, Error diagnostics -> errors <- errors @ diagnostics | TypeInfer.TSLetRecGroup(bindings, isExported, groupSpan) -> + let statement = TypeInfer.TSLetRecGroup(bindings, isExported, groupSpan) for name, expression, inferred, span in bindings do match lowerExpression state topLevel inferred expression with | Ok value -> @@ -534,6 +546,7 @@ module HirLowering = [ ({ Id = Map.find name topLevel Name = name Scheme = schemeFor (Map.find name topLevel) (Types.generalize Map.empty inferred) + Attributes = attributesFor statement IsRecursive = true IsExported = isExported Export = (exportMetadata name span isExported) @@ -541,10 +554,12 @@ module HirLowering = Value = value }: Hir.Declaration) ] | Error diagnostics -> errors <- errors @ diagnostics | TypeInfer.TSBackendImport(name, importedType, import, span) -> + let statement = TypeInfer.TSBackendImport(name, importedType, import, span) declarations <- declarations @ [ ({ Id = Map.find name topLevel Name = name Scheme = schemeFor (Map.find name topLevel) (Forall([], importedType)) + Attributes = attributesFor statement IsRecursive = false IsExported = false Export = None @@ -566,6 +581,7 @@ module HirLowering = Name = definition.Name IsRecursive = definition.IsRecursive Definition = state.TypeDefinitions.[definition.Name] + Attributes = attributesFor (TypeInfer.TSType definition) Import = importedRecords |> List.tryFind (fun imported -> imported.Name = definition.Name) Span = definition.Span }: Hir.TypeDeclaration)) let diagnostics = errors @ state.Diagnostics diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index 0bfff98..0ba4e78 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -34,10 +34,18 @@ module HirValidation = | TUnion(name, _) -> name <> declaration.Name | _ -> true || declaration.Import |> Option.exists (fun imported -> imported.Name <> declaration.Name)) + let invalidAttributes = + [ yield! program.Types |> List.collect _.Attributes + yield! program.Declarations |> List.collect _.Attributes ] + |> List.exists (fun attribute -> System.String.IsNullOrWhiteSpace attribute.Name || not (validSpan attribute.Span)) + || program.Declarations |> List.exists (fun declaration -> + declaration.IsExported <> (declaration.Attributes |> List.exists (fun attribute -> attribute.Name = "Export"))) let invalidImports = program.ImportedRecords |> List.exists (fun record -> System.String.IsNullOrWhiteSpace(record.ModulePath) || System.String.IsNullOrWhiteSpace(record.MemberName)) || (program.ImportedRecords |> List.map _.Name |> List.distinct |> List.length) <> program.ImportedRecords.Length - if invalidTypes then + if invalidAttributes then + Error [ Diagnostic.hir "FSH1108" "HIR structured attribute metadata is invalid or inconsistent." None ] + elif invalidTypes then Error [ Diagnostic.hir "FSH1107" "HIR type declarations must be unique, well-formed, and consistently linked." None ] elif invalidImports then Error [ Diagnostic.hir "FSH1105" "HIR imported record linkage must be complete and uniquely named." None ] diff --git a/src/FScript.Language/Parser.fs b/src/FScript.Language/Parser.fs index 269794a..66ec981 100644 --- a/src/FScript.Language/Parser.fs +++ b/src/FScript.Language/Parser.fs @@ -1617,7 +1617,7 @@ module Parser = hasAttributes <- true let attribute = parseAttribute() match attribute.Name with - | "export" -> + | "export" | "Export" -> if isExported then raise (ParseException { Message = "Duplicate attribute 'export'"; Span = attribute.Span }) if not attribute.Arguments.IsEmpty then diff --git a/src/FScript.Language/TypeInfer.fs b/src/FScript.Language/TypeInfer.fs index 211d0f6..077ce98 100644 --- a/src/FScript.Language/TypeInfer.fs +++ b/src/FScript.Language/TypeInfer.fs @@ -225,6 +225,7 @@ module TypeInfer = type ResolvedStatement = { Source: TypedStmt + Attributes: Attribute list Export: ResolvedExport option Expressions: ResolvedExpression list } @@ -1701,7 +1702,18 @@ module TypeInfer = | TSLetPattern(_, _, _, true, span) -> Some { Name = ""; Span = span } | TSLetRecGroup(bindings, true, span) -> Some { Name = bindings |> List.map (fun (name, _, _, _) -> name) |> String.concat ","; Span = span } | _ -> None - { Source = statement; Export = export; Expressions = List.map resolveExpression roots }) + let attributes = + match statement with + | TSLet(_, _, _, _, true, span) | TSLetPattern(_, _, _, true, span) | TSLetRecGroup(_, true, span) -> + [ { Name = "Export"; Arguments = []; Span = span } ] + | TSBackendImport(_, _, import, _) -> + [ { Name = "Import"; Arguments = [ import.ModulePath; import.MemberName ]; Span = import.Span } ] + | TSType definition -> + definition.Import + |> Option.map (fun import -> [ { Name = "Import"; Arguments = [ import.ModulePath; import.MemberName ]; Span = import.Span } ]) + |> Option.defaultValue [] + | _ -> [] + { Source = statement; Attributes = attributes; Export = export; Expressions = List.map resolveExpression roots }) { TypedProgram = typed ResolvedStatements = resolvedStatements ExpressionTypes = expressions diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index 1dd3d6e..c6838e5 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -97,12 +97,20 @@ type HirTests() = let identity = compiled.Hir.Declarations |> List.find (fun declaration -> declaration.Name = "identity") Assert.That(identity.Scheme, Is.EqualTo(Forall([ 0 ], TFun(TVar 0, TVar 0)))) + [] + member _.``HIR attaches structured export and import attributes`` () = + let source = "[] type Person = { Name: string }\n[] let answer = 42" + let compiled = match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with Ok value -> value | Error diagnostics -> failwith (System.String.Join("; ", diagnostics |> List.map _.Message)) + Assert.That((compiled.Hir.Types.Head.Attributes |> List.map _.Name) = [ "Import" ], Is.True) + let answer = compiled.Hir.Declarations |> List.find (fun declaration -> declaration.Name = "answer") + Assert.That((answer.Attributes |> List.map _.Name) = [ "Export" ], Is.True) + [] member _.``HIR validation rejects a declaration scheme mismatch`` () = let value: Hir.Expression = { Type = TInt; Span = span; Node = Hir.Constant(Hir.Int 1L) } let declaration: Hir.Declaration = { Id = Hir.SymbolId 0; Name = "value"; Scheme = Forall([], TString) - IsRecursive = false; IsExported = false; Export = None; Span = span; Value = value } + Attributes = []; IsRecursive = false; IsExported = false; Export = None; Span = span; Value = value } match HirValidation.validate { Types = []; ImportedRecords = []; Declarations = [ declaration ]; Entry = None } with | Ok _ -> Assert.Fail("Expected invalid HIR diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSH1104")) diff --git a/tests/FScript.Language.Tests/ParserTests.fs b/tests/FScript.Language.Tests/ParserTests.fs index cf85107..2cde0aa 100644 --- a/tests/FScript.Language.Tests/ParserTests.fs +++ b/tests/FScript.Language.Tests/ParserTests.fs @@ -801,9 +801,10 @@ type ParserTests () = act |> should throw typeof [] - member _.``Rejects wrong attribute casing`` () = - let act () = Helpers.parse "[] let y = 1" |> ignore - act |> should throw typeof + member _.``Parses canonical Export attribute casing`` () = + match Helpers.parse "[] let y = 1" with + | [ SLet("y", _, _, _, _, true, _) ] -> () + | _ -> Assert.Fail("Expected exported let binding") [] member _.``Rejects export attribute in nested expression block`` () = diff --git a/tests/FScript.Language.Tests/ResolvedInferenceTests.fs b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs index f8ed7eb..3eaaba4 100644 --- a/tests/FScript.Language.Tests/ResolvedInferenceTests.fs +++ b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs @@ -77,9 +77,19 @@ type ResolvedInferenceTests() = [] member _.``Resolved statements preserve structured export metadata`` () = - let resolved = Helpers.parse "[] let answer = 42" |> TypeInfer.inferResolvedProgram + let resolved = Helpers.parse "[] let answer = 42" |> TypeInfer.inferResolvedProgram match resolved.ResolvedStatements |> List.tryPick _.Export with | Some export -> Assert.That(export.Name, Is.EqualTo("answer")) Assert.That(export.Span.Start.Line, Is.EqualTo(1)) | None -> Assert.Fail("Expected resolved export metadata") + let attributes = resolved.ResolvedStatements |> List.collect _.Attributes + Assert.That(attributes |> List.map _.Name, Does.Contain("Export")) + + [] + member _.``Resolved statements attach structured import attributes`` () = + let source = "[] type Person = { Name: string }\n[] let upper : string -> string" + let resolved = Helpers.parse source |> TypeInfer.inferResolvedProgram + let imports = resolved.ResolvedStatements |> List.collect _.Attributes |> List.filter (fun attribute -> attribute.Name = "Import") + Assert.That(imports.Length, Is.EqualTo(2)) + Assert.That(imports |> List.map _.Arguments, Does.Contain([ "strings"; "ToUpper" ])) From 72eeed9b07d52d8133d44ba33980061b6b21cf69 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 20:21:09 +0200 Subject: [PATCH 049/133] feat(compiler): preserve default runtime extern linkage --- CHANGELOG.md | 2 + docs/specs/compiler-semantics-baseline.md | 14 +++++++ src/FScript.Compiler/Compiler.fs | 17 +------- src/FScript.Compiler/Diagnostics.fs | 41 +++++++++++++++++++ .../GoBackendTests.fs | 15 +++++++ tests/FScript.Compiler.Tests/HirTests.fs | 27 ++++++++++-- .../FScript.Compiler.Tests/Snapshots/add.hir | 16 ++++---- .../FScript.Compiler.Tests/Snapshots/add.lir | 18 ++++---- .../LirInterpreterTests.fs | 15 +++++++ 9 files changed, 128 insertions(+), 37 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 60eb0f7..b2fd70e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Compilation now preserves every default runtime external as typed host linkage with reachability-based backend diagnostics. + - Attached normalized structured Export and Import attributes to resolved statements and HIR declarations. - Consumed normalized frontend declaration schemes in HIR and expanded cross-runtime aggregate, closure, record, union, and map conformance. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 47c8f8a..d902bb6 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -65,6 +65,20 @@ Compiled maps use an ordered entry representation for string and integer keys. E - `[]` is declaration metadata used by embedding hosts and does not change inference. Backend `Import` metadata is separate and will not weaken the declared FScript type. - Both `[]` and `[]` normalize to a structured `Export` attribute in resolved statements and HIR. `Import` attributes retain their two arguments and attribute span on ordinary HIR function/type declarations in addition to dedicated backend linkage. +The compiler-owned runtime catalog is declarative and contains no executable host delegates: + +| Classification | Current bindings | +| --- | --- | +| Compiler context intrinsic | `Env` | +| Portable runtime intrinsic | `Console.writeLine` | +| Deferred host input | `Console.readLine` | +| Host filesystem | `Fs.readText`, `Fs.exists`, `Fs.kind`, `Fs.createDirectory`, `Fs.writeText`, `Fs.glob`, `Fs.enumerateFiles` | +| Pure host candidates | `Fs.combinePath`, `Fs.parentDirectory`, `Fs.extension`, `Fs.fileNameWithoutExtension`, `Regex.matchGroups`, `Hash.md5` | +| Host-only/deferred data services | `Guid.new`, `Json.deserialize`, `Json.serialize`, `Xml.queryValues` | +| Deferred task runtime | `Task.spawn`, `Task.await` | + +All entries are type-checkable and retained as linkage in HIR/LIR. Except for the compiler context and portable intrinsic rows, reaching one produces an interpreter or backend unsupported diagnostic; unused linkage remains inert. + ## Diagnostics Parse, type, and evaluation failures currently use source spans and exceptions. Compiler entry points translate expected failures into structured stage diagnostics while leaving existing embedding APIs behavior-compatible. diff --git a/src/FScript.Compiler/Compiler.fs b/src/FScript.Compiler/Compiler.fs index a4d2b4b..6170375 100644 --- a/src/FScript.Compiler/Compiler.fs +++ b/src/FScript.Compiler/Compiler.fs @@ -11,24 +11,9 @@ type CompiledProgram = Lir: Lir.Program } module Compiler = - let private descriptor name scheme arity kind : CompilerExtern = - { Name = name; Scheme = scheme; Arity = arity; Kind = kind } - - let private runtimeContext : CompilerExtern = - descriptor "Env" (Forall([], TNamed "Environment")) 0 RuntimeContextIntrinsic - - let private taskBindings = - [ descriptor "Task.spawn" (Forall([ 0 ], TFun(TFun(TUnit, TVar 0), TTask(TVar 0)))) 1 UnsupportedHostExtern - descriptor "Task.await" (Forall([ 0 ], TFun(TTask(TVar 0), TVar 0))) 1 UnsupportedHostExtern ] - - let private runtimeIntrinsics = - RuntimeIntrinsics.schemes - |> Map.toList - |> List.map (fun (name, scheme) -> descriptor name scheme 1 PortableRuntimeIntrinsic) - let compile input = try - let compilerBindings = runtimeContext :: (runtimeIntrinsics @ taskBindings) + let compilerBindings = CompilerExternCatalog.defaults let reserved = compilerBindings |> List.map _.Name |> Set.ofList let externs = compilerBindings @ (input.Externs |> List.filter (fun external -> not (Set.contains external.Name reserved))) let inferenceExterns = diff --git a/src/FScript.Compiler/Diagnostics.fs b/src/FScript.Compiler/Diagnostics.fs index 12b9f81..e4af39a 100644 --- a/src/FScript.Compiler/Diagnostics.fs +++ b/src/FScript.Compiler/Diagnostics.fs @@ -31,6 +31,47 @@ module RuntimeIntrinsics = Map.ofList [ "Console.writeLine", Forall([], TFun(TString, TUnit)) ] +/// Declarative descriptions of the runtime surface known to the compiler. +/// These values deliberately contain no executable .NET delegates. +module CompilerExternCatalog = + let private descriptor name scheme arity kind : CompilerExtern = + { Name = name + Scheme = scheme + Arity = arity + Kind = kind } + + let runtimeContext = + descriptor "Env" (Forall([], TNamed "Environment")) 0 RuntimeContextIntrinsic + + let portableRuntime = + RuntimeIntrinsics.schemes + |> Map.toList + |> List.map (fun (name, scheme) -> descriptor name scheme 1 PortableRuntimeIntrinsic) + + let unsupportedHostRuntime = + [ descriptor "Console.readLine" (Forall([], TFun(TUnit, TOption TString))) 1 UnsupportedHostExtern + descriptor "Fs.readText" (Forall([], TFun(TString, TOption TString))) 1 UnsupportedHostExtern + descriptor "Fs.exists" (Forall([], TFun(TString, TBool))) 1 UnsupportedHostExtern + descriptor "Fs.kind" (Forall([], TFun(TString, TNamed "FsKind"))) 1 UnsupportedHostExtern + descriptor "Fs.createDirectory" (Forall([], TFun(TString, TBool))) 1 UnsupportedHostExtern + descriptor "Fs.writeText" (Forall([], TFun(TString, TFun(TString, TBool)))) 2 UnsupportedHostExtern + descriptor "Fs.combinePath" (Forall([], TFun(TString, TFun(TString, TString)))) 2 UnsupportedHostExtern + descriptor "Fs.parentDirectory" (Forall([], TFun(TString, TOption TString))) 1 UnsupportedHostExtern + descriptor "Fs.extension" (Forall([], TFun(TString, TOption TString))) 1 UnsupportedHostExtern + descriptor "Fs.fileNameWithoutExtension" (Forall([], TFun(TString, TString))) 1 UnsupportedHostExtern + descriptor "Fs.glob" (Forall([], TFun(TString, TOption(TList TString)))) 1 UnsupportedHostExtern + descriptor "Fs.enumerateFiles" (Forall([], TFun(TString, TFun(TString, TOption(TList TString))))) 2 UnsupportedHostExtern + descriptor "Regex.matchGroups" (Forall([], TFun(TString, TFun(TString, TOption(TList TString))))) 2 UnsupportedHostExtern + descriptor "Hash.md5" (Forall([], TFun(TString, TOption TString))) 1 UnsupportedHostExtern + descriptor "Guid.new" (Forall([ 0 ], TFun(TVar 0, TOption TString))) 1 UnsupportedHostExtern + descriptor "Json.deserialize" (Forall([ 0 ], TFun(TTypeToken, TFun(TString, TOption(TVar 0))))) 2 UnsupportedHostExtern + descriptor "Json.serialize" (Forall([ 0 ], TFun(TVar 0, TOption TString))) 1 UnsupportedHostExtern + descriptor "Xml.queryValues" (Forall([], TFun(TString, TFun(TString, TOption(TList TString))))) 2 UnsupportedHostExtern + descriptor "Task.spawn" (Forall([ 0 ], TFun(TFun(TUnit, TVar 0), TTask(TVar 0)))) 1 UnsupportedHostExtern + descriptor "Task.await" (Forall([ 0 ], TFun(TTask(TVar 0), TVar 0))) 1 UnsupportedHostExtern ] + + let defaults = runtimeContext :: (portableRuntime @ unsupportedHostRuntime) + module Diagnostic = let hir code message span = { Code = code diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index 871e482..cb01968 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -48,6 +48,21 @@ type GoBackendTests() = | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) | Ok _ -> () + [] + member _.``Go backend reports default host runtime externs only when reachable`` () = + let compile source = + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Ok value -> value.Lir + | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + + match GoBackend.generate { ModuleName = "fscript.generated/runtimeexterns" } (compile "let unused = Fs.exists\n1") with + | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) + | Ok _ -> () + + match GoBackend.generate { ModuleName = "fscript.generated/runtimeexterns" } (compile "Fs.exists \"missing\"") with + | Ok _ -> Assert.Fail("Expected unsupported host-runtime diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSB4006")) + [] member _.``Go backend emits imported record boundary adapters`` () = let source = "[] type Person = { Name: string; Age: int }\n[] let birthday : Person -> Person\nbirthday { Name = \"Ada\"; Age = 42 }" diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index c6838e5..3f168ae 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -14,6 +14,27 @@ type HirTests() = | Ok value -> value | Error diagnostics -> failwith (System.String.Join("; ", diagnostics |> List.map _.Message)) + [] + member _.``Compiler catalog describes every default runtime external without delegates`` () = + let expected = + [ "Console.readLine"; "Console.writeLine"; "Env" + "Fs.combinePath"; "Fs.createDirectory"; "Fs.enumerateFiles"; "Fs.exists" + "Fs.extension"; "Fs.fileNameWithoutExtension"; "Fs.glob"; "Fs.kind" + "Fs.parentDirectory"; "Fs.readText"; "Fs.writeText" + "Guid.new"; "Hash.md5"; "Json.deserialize"; "Json.serialize" + "Regex.matchGroups"; "Task.await"; "Task.spawn"; "Xml.queryValues" ] + Assert.That((CompilerExternCatalog.defaults |> List.map _.Name |> List.sort) = expected, Is.True) + + match Compiler.compile { Program = Parser.parseProgram "Fs.exists \"missing\""; Externs = [] } with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok compiled -> + Assert.That( + compiled.Lir.Declarations + |> List.exists (fun declaration -> + declaration.Name = "Fs.exists" + && match declaration.Value.Node with Lir.HostExternal("Fs.exists", 1) -> true | _ -> false), + Is.True) + [] member _.``Lowers deterministic typed add program`` () = let source = "let add x y = x + y\nadd 1 2" @@ -59,10 +80,8 @@ type HirTests() = let program = Parser.parseProgram source let external name scheme arity = ({ Name = name; Scheme = scheme; Arity = arity; Impl = fun _ _ -> VUnit }: ExternalFunction) let frontendExterns = - [ external "Env" (Forall([], TNamed "Environment")) 0 - yield! RuntimeIntrinsics.schemes |> Map.toList |> List.map (fun (name, scheme) -> external name scheme 1) - external "Task.spawn" (Forall([ 0 ], TFun(TFun(TUnit, TVar 0), TTask(TVar 0)))) 1 - external "Task.await" (Forall([ 0 ], TFun(TTask(TVar 0), TVar 0))) 1 ] + CompilerExternCatalog.defaults + |> List.map (fun descriptor -> external descriptor.Name descriptor.Scheme descriptor.Arity) let resolved = TypeInfer.inferResolvedProgramWithExterns frontendExterns program let compiled = match Compiler.compile { Program = program; Externs = [] } with Ok value -> value | Error diagnostics -> failwith (System.String.Join("; ", diagnostics |> List.map _.Message)) let resolvedId kind name = resolved.Symbols |> List.find (fun symbol -> symbol.Kind = kind && symbol.Name = name) |> _.SymbolId diff --git a/tests/FScript.Compiler.Tests/Snapshots/add.hir b/tests/FScript.Compiler.Tests/Snapshots/add.hir index 4ef0de8..d4d3176 100644 --- a/tests/FScript.Compiler.Tests/Snapshots/add.hir +++ b/tests/FScript.Compiler.Tests/Snapshots/add.hir @@ -1,14 +1,14 @@ -decl %54 add: ('a -> ('a -> int)) - lambda %55 x: 'a : ('a -> ('a -> int)) - lambda %56 y: 'a : ('a -> int) +decl %72 add: ('a -> ('a -> int)) + lambda %73 x: 'a : ('a -> ('a -> int)) + lambda %74 y: 'a : ('a -> int) primitive IntAdd : int - ref %55 : 'a - ref %56 : 'a -decl %57 $expression57: int + ref %73 : 'a + ref %74 : 'a +decl %75 $expression75: int apply : int apply : (int -> int) - ref %54 : (int -> (int -> int)) + ref %72 : (int -> (int -> int)) const int 1 : int const int 2 : int entry - ref %57 : int + ref %75 : int diff --git a/tests/FScript.Compiler.Tests/Snapshots/add.lir b/tests/FScript.Compiler.Tests/Snapshots/add.lir index 698fccf..f097ae0 100644 --- a/tests/FScript.Compiler.Tests/Snapshots/add.lir +++ b/tests/FScript.Compiler.Tests/Snapshots/add.lir @@ -1,16 +1,16 @@ -function @0 %55 x captures=[]: 'a -> ('a -> int) - make-closure @1 captures=[55] : ('a -> int) -function @1 %56 y captures=[55]: 'a -> int +function @0 %73 x captures=[]: 'a -> ('a -> int) + make-closure @1 captures=[73] : ('a -> int) +function @1 %74 y captures=[73]: 'a -> int operation IntAdd : int - load %55 : 'a - load %56 : 'a -decl %54 add: ('a -> ('a -> int)) + load %73 : 'a + load %74 : 'a +decl %72 add: ('a -> ('a -> int)) make-closure @0 captures=[] : ('a -> ('a -> int)) -decl %57 $expression57: int +decl %75 $expression75: int call-closure : int call-closure : (int -> int) - load %54 : (int -> (int -> int)) + load %72 : (int -> (int -> int)) const Int 1L : int const Int 2L : int entry - load %57 : int + load %75 : int diff --git a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs index fed6ae0..a1394ff 100644 --- a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs +++ b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs @@ -150,6 +150,21 @@ type LirInterpreterTests() = | Ok _ -> Assert.Fail("Expected unsupported external diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSI3021")) + [] + member _.``Default host runtime externs remain inert until reached`` () = + let execute source = + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Error diagnostics + | Ok compiled -> LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir + + match execute "let unused = Fs.exists\n42" with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 42L)) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + + match execute "Fs.exists \"missing\"" with + | Ok _ -> Assert.Fail("Expected unsupported host-runtime diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSI3021")) + [] member _.``Retains tasks in LIR and rejects them only when reached`` () = let compile source = Compiler.compile { Program = Parser.parseProgram source; Externs = [] } From 05d24769e613eb0aa833c5ff58be92718518e79f Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 20:40:15 +0200 Subject: [PATCH 050/133] refactor(language): emit semantic resolved trees --- CHANGELOG.md | 2 + docs/architecture/typed-ir-and-backends.md | 4 +- docs/specs/compiler-semantics-baseline.md | 2 +- src/FScript.Compiler/HirLowering.fs | 728 ++++++------------ src/FScript.Language/TypeInfer.fs | 444 +++++++++-- tests/FScript.Compiler.Tests/HirTests.fs | 18 + .../ConformanceTests.fs | 1 + .../ResolvedInferenceTests.fs | 22 + 8 files changed, 690 insertions(+), 531 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b2fd70e..5421d49 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- HIR lowering now consumes parser-independent semantic resolved statements, expressions, patterns, references, and declaration schemes. + - Compilation now preserves every default runtime external as typed host linkage with reachability-based backend diagnostics. - Attached normalized structured Export and Import attributes to resolved statements and HIR declarations. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 352ffd4..b6603a4 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -10,7 +10,7 @@ source -> parser/includes -> inference + resolved typed tree -> HIR -> LIR +-> Go source backend ``` -`FScript.Language` remains the parser, type-system, legacy evaluator, and Fable-compatible package. It emits a recursive resolved typed tree in the same inference pass as its compatibility `TypedProgram`. Every resolved expression carries its final instantiated type and span plus optional lexical-reference, primitive-operation, and structural/named-record conversion evidence; compatibility inference and evaluation APIs continue to project their existing results. +`FScript.Language` remains the parser, type-system, legacy evaluator, and Fable-compatible package. It emits a recursive semantic resolved tree in the same inference pass as its compatibility `TypedProgram`. Resolved statement, expression, and pattern nodes do not contain parser AST nodes; every resolved expression carries its final instantiated type and span plus lexical-reference kind, primitive operation, and structural/named-record conversion evidence. Compatibility inference and evaluation APIs continue to project their existing results. Resolved inference assigns deterministic program-local IDs to builtins, externs, types, union cases, declarations, parameters, pattern bindings, and locals in normalized source traversal. Declaration schemes and instantiated reference types normalize type variables from zero, so results do not depend on the process-global fresh-variable counter. The legacy `TypedProgram` remains a projection with unchanged public entry points. @@ -20,7 +20,7 @@ Validated LIR consumers are separate leaf assemblies. `FScript.Interpreter.Lir` ## IR boundaries -HIR is language-oriented and contains resolved artifact-local symbol identities, ordinary and imported type declarations, complete semantic types and schemes, source locations, structured export metadata, and explicit nodes for named/structural record conversions. HIR directly reuses the frontend IDs for builtins, externs, types, constructors, declarations, parameters, pattern bindings, and locals; only compiler-synthesized nodes allocate IDs above the frontend range. Named record and union references remain nominal in HIR and LIR; their declaration shapes are carried separately and serialized in artifacts. Export metadata retains the exported source name and span through LIR and artifacts while the legacy Boolean remains available to compatibility consumers. Conversion evidence is produced by inference, not rediscovered from backend representations. HIR contains no Go types. +HIR is language-oriented and contains resolved artifact-local symbol identities, ordinary and imported type declarations, complete semantic types and schemes, source locations, structured export metadata, and explicit nodes for named/structural record conversions. HIR lowering consumes only the semantic resolved statements, expressions, and patterns; it does not traverse `Expr`, `Pattern`, `TypedStmt`, or compatibility expression telemetry. HIR directly reuses the frontend IDs for builtins, externs, types, constructors, declarations, parameters, pattern bindings, and locals; only compiler-synthesized nodes allocate IDs above the frontend range. Named record and union references remain nominal in HIR and LIR; their declaration shapes are carried separately and serialized in artifacts. Export metadata retains the exported source name and span through LIR and artifacts while the legacy Boolean remains available to compatibility consumers. Conversion evidence is produced by inference, not rediscovered from backend representations. HIR contains no Go types. LIR is executable but not a Go AST. Lambda bodies are lifted into a deterministic program-level function table. Closure creation references a function ID and carries its canonical ordered set of captured symbol IDs, alongside resolved primitive operations, aggregate construction/projection, and structured control flow. Validation recomputes each lifted function's free-symbol set and rejects unknown function IDs or forged, missing, duplicate, or unordered captures. Pattern matching becomes deterministic tests and projections before reaching a backend. Semantic types remain alongside selected runtime representations for validation and diagnostics. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index d902bb6..7ae126f 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -9,7 +9,7 @@ This document records the behavior that the typed IR, LIR interpreter, and compi - Includes are resolved into one flattened program before inference. Alias-qualified declarations are rewritten by the include resolver; unresolved `SImport` nodes are rejected by inference. - Hindley-Milner inference generalizes top-level and local let bindings after applying substitutions. Recursive bindings are inferred monomorphically inside their recursive group and generalized afterward. - The active typed program stores top-level types but retains parser expressions in bodies. Compiler lowering therefore requires a new resolved, recursively typed result emitted by the inference pass; types cannot safely be reconstructed afterward. -- Resolved inference exposes a recursive expression tree whose nodes carry their final instantiated type, span, lexical reference, resolved primitive operation, and named/structural record-coercion evidence. It also exposes deterministic symbols for every lexical binding category and links each captured reference to its target ID. Type-variable numbering in resolved schemes and reference types is normalized independently of prior inference calls. +- Resolved inference exposes parser-independent semantic statement, expression, and pattern trees whose nodes carry their final instantiated type, span, lexical reference kind, resolved primitive operation, and named/structural record-coercion evidence. It also exposes deterministic symbols for every lexical binding category and links each captured reference to its target ID. Repeated occurrences of one pattern binding reuse one ID so equality constraints cannot be lost during lowering. Type-variable numbering in resolved schemes and reference types is normalized independently of prior inference calls. - HIR reuses those resolved IDs unchanged for source symbols even when type and value declarations are interleaved; compiler-synthesized temporaries and constructor payload parameters occupy IDs above the resolved frontend range. - Top-level statements execute in source order. Every non-final expression must have type `unit`; the final expression is the program result. - Compiled top-level expressions execute exactly once in source order even when interleaved with declarations. Top-level pattern bindings evaluate their scrutinee once, fail before later statements when unmatched, and expose all bound names to subsequent statements. diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index 2253a12..1a2053a 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -5,10 +5,6 @@ open FScript.Language module HirLowering = type private State = { mutable NextSymbol: int - ExpressionTypes: Map - Coercions: Map - mutable UnionCases: Map - mutable TypeDefinitions: Map mutable AvailableSymbols: TypeInfer.ResolvedSymbol list mutable Diagnostics: CompilerDiagnostic list } @@ -40,348 +36,144 @@ module HirLowering = | TMap(key, value) -> TMap(representationType state key, representationType state value) | TTask item -> TTask(representationType state item) | TRecord fields -> TRecord(fields |> Map.map (fun _ -> representationType state)) + | TUnion(name, cases) -> TUnion(name, cases |> Map.map (fun _ -> Option.map (representationType state))) | TNamed name -> TNamed name | value -> value - let private resolvedDefinition state = function - | TNamed name as named -> state.TypeDefinitions |> Map.tryFind name |> Option.defaultValue named - | value -> value - - let private recordFields state value = - match resolvedDefinition state value with - | TRecord fields -> fields - | _ -> Map.empty - - let private expressionType state expression fallback = - state.ExpressionTypes - |> Map.tryFind expression - |> Option.defaultValue fallback - |> representationType state - - let private literal = function - | LInt value -> Hir.Int value, TInt - | LFloat value -> Hir.Float value, TFloat - | LBool value -> Hir.Bool value, TBool - | LString value -> Hir.String value, TString + let private resolvedOperation = function + | TypeInfer.IntegerAddition -> Hir.IntAdd + | TypeInfer.FloatingAddition -> Hir.FloatAdd + | TypeInfer.IntegerSubtraction -> Hir.IntSubtract + | TypeInfer.FloatingSubtraction -> Hir.FloatSubtract + | TypeInfer.IntegerMultiplication -> Hir.IntMultiply + | TypeInfer.FloatingMultiplication -> Hir.FloatMultiply + | TypeInfer.IntegerDivision -> Hir.IntDivide + | TypeInfer.FloatingDivision -> Hir.FloatDivide + | TypeInfer.IntegerRemainder -> Hir.IntRemainder + | TypeInfer.StructuralEquality -> Hir.StructuralEqual + | TypeInfer.NumericComparison "<" -> Hir.NumericLessThan + | TypeInfer.NumericComparison ">" -> Hir.NumericGreaterThan + | TypeInfer.NumericComparison "<=" -> Hir.NumericLessThanOrEqual + | TypeInfer.NumericComparison ">=" -> Hir.NumericGreaterThanOrEqual + | TypeInfer.BooleanOperation "&&" -> Hir.BooleanAnd + | TypeInfer.BooleanOperation "||" -> Hir.BooleanOr + | value -> invalidOp $"Unsupported resolved primitive operation '{value}'." - let private operation op operandType = - match op, operandType with - | "+", TInt -> Some Hir.IntAdd - | "+", TFloat -> Some Hir.FloatAdd - | "-", TInt -> Some Hir.IntSubtract - | "-", TFloat -> Some Hir.FloatSubtract - | "*", TInt -> Some Hir.IntMultiply - | "*", TFloat -> Some Hir.FloatMultiply - | "/", TInt -> Some Hir.IntDivide - | "/", TFloat -> Some Hir.FloatDivide - | "%", TInt -> Some Hir.IntRemainder - | "=", _ -> Some Hir.StructuralEqual - | "<", _ -> Some Hir.NumericLessThan - | ">", _ -> Some Hir.NumericGreaterThan - | "<=", _ -> Some Hir.NumericLessThanOrEqual - | ">=", _ -> Some Hir.NumericGreaterThanOrEqual - | "&&", TBool -> Some Hir.BooleanAnd - | "||", TBool -> Some Hir.BooleanOr - | _ -> None + let private resolvedLiteral = function + | TypeInfer.ResolvedUnit -> Hir.Unit + | TypeInfer.ResolvedInt value -> Hir.Int value + | TypeInfer.ResolvedFloat value -> Hir.Float value + | TypeInfer.ResolvedBool value -> Hir.Bool value + | TypeInfer.ResolvedString value -> Hir.String value - let private collectResults<'a> (results: Result<'a, CompilerDiagnostic list> list) : Result<'a list, CompilerDiagnostic list> = - let folder state result = - match state, result with - | Ok values, Ok value -> Ok(values @ [ value ]) - | Error a, Error b -> Error(a @ b) - | Error errors, _ | _, Error errors -> Error errors - results |> List.fold folder (Ok []) + let rec private lowerResolvedPattern state (pattern: TypeInfer.ResolvedPattern) : Hir.Pattern * Map = + let make node = ({ Type = representationType state pattern.Type; Span = pattern.Span; Node = node }: Hir.Pattern) + match pattern.Node with + | TypeInfer.ResolvedWildcard -> make Hir.Wildcard, Map.empty + | TypeInfer.ResolvedPatternBinding binding -> + let id = Hir.SymbolId binding.SymbolId + make (Hir.Binding(id, binding.Name)), Map.ofList [ binding.Name, id ] + | TypeInfer.ResolvedLiteralPattern value -> make (Hir.LiteralPattern(resolvedLiteral value)), Map.empty + | TypeInfer.ResolvedEmptyList -> make Hir.EmptyList, Map.empty + | TypeInfer.ResolvedNonePattern -> make Hir.NonePattern, Map.empty + | TypeInfer.ResolvedSomePattern item -> + let lowered, bindings = lowerResolvedPattern state item + make (Hir.SomePattern lowered), bindings + | TypeInfer.ResolvedConsPattern(head, tail) -> + let loweredHead, headBindings = lowerResolvedPattern state head + let loweredTail, tailBindings = lowerResolvedPattern state tail + make (Hir.ConsPattern(loweredHead, loweredTail)), Map.fold (fun env name id -> Map.add name id env) headBindings tailBindings + | TypeInfer.ResolvedTuplePattern items -> + let values = List.map (lowerResolvedPattern state) items + make (Hir.TuplePattern(List.map fst values)), values |> List.map snd |> List.fold (Map.fold (fun env name id -> Map.add name id env)) Map.empty + | TypeInfer.ResolvedRecordPattern fields -> + let values = fields |> List.map (fun (name, item) -> let lowered, bindings = lowerResolvedPattern state item in (name, lowered), bindings) + make (Hir.RecordPattern(List.map fst values)), values |> List.map snd |> List.fold (Map.fold (fun env name id -> Map.add name id env)) Map.empty + | TypeInfer.ResolvedMapPattern(clauses, tail) -> + let values = clauses |> List.map (fun (key, value) -> + let loweredKey, keyBindings = lowerResolvedPattern state key + let loweredValue, valueBindings = lowerResolvedPattern state value + (loweredKey, loweredValue), Map.fold (fun env name id -> Map.add name id env) keyBindings valueBindings) + let loweredTail, tailBindings = + match tail with + | Some item -> let lowered, bindings = lowerResolvedPattern state item in Some lowered, bindings + | None -> None, Map.empty + let bindings = tailBindings :: List.map snd values |> List.fold (Map.fold (fun env name id -> Map.add name id env)) Map.empty + make (Hir.MapPattern(List.map fst values, loweredTail)), bindings + | TypeInfer.ResolvedUnionPattern(typeName, caseName, payload) -> + match payload with + | Some item -> + let lowered, bindings = lowerResolvedPattern state item + make (Hir.UnionPattern(typeName, caseName, Some lowered)), bindings + | None -> make (Hir.UnionPattern(typeName, caseName, None)), Map.empty + | TypeInfer.ResolvedTypePattern patternType -> make (Hir.TypePattern(representationType state patternType)), Map.empty - let rec private lowerPattern state expected pattern = - let span = Ast.spanOfPattern pattern - let make node = ({ Type = expected; Span = span; Node = node }: Hir.Pattern) - match pattern with - | PWildcard _ -> Ok(make Hir.Wildcard, Map.empty) - | PVar(name, _) -> - let id = claimSymbol state TypeInfer.PatternBindingSymbol name span - Ok(make (Hir.Binding(id, name)), Map.ofList [ name, id ]) - | PLiteral(value, _) -> let lowered, _ = literal value in Ok(make (Hir.LiteralPattern lowered), Map.empty) - | PNil _ -> Ok(make Hir.EmptyList, Map.empty) - | PNone _ -> Ok(make Hir.NonePattern, Map.empty) - | PSome(inner, _) -> - let itemType = match expected with TOption value -> value | _ -> Types.freshVar() - lowerPattern state itemType inner |> Result.map (fun (value, bindings) -> make (Hir.SomePattern value), bindings) - | PCons(head, tail, _) -> - let itemType = match expected with TList value -> value | _ -> Types.freshVar() - match lowerPattern state itemType head, lowerPattern state expected tail with - | Ok(h, hb), Ok(t, tb) -> Ok(make (Hir.ConsPattern(h, t)), Map.fold (fun env key value -> Map.add key value env) hb tb) - | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e - | PTuple(items, _) -> - let types = match expected with TTuple values when values.Length = items.Length -> values | _ -> List.replicate items.Length (Types.freshVar()) - List.zip items types |> List.map (fun (item, itemType) -> lowerPattern state itemType item) - |> collectResults |> Result.map (fun values -> - let patterns, bindingMaps = List.unzip values - let bindings = bindingMaps |> List.fold (Map.fold (fun env key value -> Map.add key value env)) Map.empty - make (Hir.TuplePattern patterns), bindings) - | PRecord(fields, _) -> - let types = recordFields state expected - fields |> List.map (fun (name, item) -> lowerPattern state (types |> Map.tryFind name |> Option.defaultValue (Types.freshVar())) item |> Result.map (fun (value, bindings) -> (name, value), bindings)) - |> collectResults |> Result.map (fun values -> - let fields, bindingMaps = List.unzip values - let bindings = bindingMaps |> List.fold (Map.fold (fun env key value -> Map.add key value env)) Map.empty - make (Hir.RecordPattern fields), bindings) - | PMap(clauses, tail, _) -> - let keyType, valueType = match expected with TMap(key, value) -> key, value | _ -> Types.freshVar(), Types.freshVar() - let loweredClauses = - clauses |> List.map (fun (keyPattern, valuePattern) -> - match lowerPattern state keyType keyPattern, lowerPattern state valueType valuePattern with - | Ok(key, keyBindings), Ok(value, valueBindings) -> - Ok((key, value), Map.fold (fun env name id -> Map.add name id env) keyBindings valueBindings) - | Error a, Error b -> Error(a @ b) | Error errors, _ | _, Error errors -> Error errors) - |> collectResults - let loweredTail = - tail |> Option.map (lowerPattern state expected >> Result.map Some) |> Option.defaultValue (Ok None) - match loweredClauses, loweredTail with - | Ok clausesWithBindings, Ok tailValue -> - let clauses, bindingMaps = List.unzip clausesWithBindings - let tailPattern, tailBindings = tailValue |> Option.map (fun (pattern, bindings) -> Some pattern, bindings) |> Option.defaultValue (None, Map.empty) - let bindings = tailBindings :: bindingMaps |> List.fold (Map.fold (fun env name id -> Map.add name id env)) Map.empty - Ok(make (Hir.MapPattern(clauses, tailPattern)), bindings) - | Error a, Error b -> Error(a @ b) | Error errors, _ | _, Error errors -> Error errors - | PUnionCase(typeName, caseName, payload, _) -> - let resolvedTypeName, payloadType = - match expected with - | TUnion(name, cases) -> name, cases |> Map.tryFind caseName |> Option.flatten - | _ -> - state.UnionCases |> Map.tryFind caseName - |> Option.defaultValue (typeName |> Option.defaultValue "", None) - match payload, payloadType with - | None, _ -> Ok(make (Hir.UnionPattern(resolvedTypeName, caseName, None)), Map.empty) - | Some item, Some itemType -> lowerPattern state itemType item |> Result.map (fun (value, bindings) -> make (Hir.UnionPattern(resolvedTypeName, caseName, Some value)), bindings) - | Some item, None -> lowerPattern state (Types.freshVar()) item |> Result.map (fun (value, bindings) -> make (Hir.UnionPattern(resolvedTypeName, caseName, Some value)), bindings) - | PTypeRef _ -> Ok(make (Hir.TypePattern expected), Map.empty) - - let rec private lowerExpression state environment fallback expression = - let span = Ast.spanOfExpr expression - let inferred = expressionType state expression fallback - let make node = ({ Type = inferred; Span = span; Node = node }: Hir.Expression) - match expression with - | EUnit _ -> Ok(make (Hir.Constant Hir.Unit)) - | ELiteral(value, _) -> - let value, _ = literal value - Ok(make (Hir.Constant value)) - | EParen(inner, _) -> lowerExpression state environment inferred inner - | EVar(name, _) -> - match environment |> Map.tryFind name with - | Some id -> Ok(make (Hir.Reference id)) - | None -> Error [ Diagnostic.hir "FSH1001" $"Unresolved HIR reference '{name}'." (Some span) ] - | ELambda(parameter, body, _) -> - match inferred with - | TFun(parameterType, resultType) -> - let id = claimSymbol state TypeInfer.ParameterSymbol parameter.Name parameter.Span - lowerExpression state (environment |> Map.add parameter.Name id) resultType body - |> Result.map (fun lowered -> make (Hir.Lambda(id, parameter.Name, parameterType, lowered))) - | _ -> Error [ Diagnostic.hir "FSH1002" "Lambda does not have a function type." (Some span) ] - | EApply(fn, argument, _) -> - let fnType = expressionType state fn (TFun(Types.freshVar(), inferred)) - let argumentType = - match fnType with - | TFun(value, _) -> value - | _ -> Types.freshVar() - match lowerExpression state environment fnType fn, lowerExpression state environment argumentType argument with - | Ok loweredFn, Ok loweredArgument -> - let argumentWithEvidence = - state.Coercions - |> Map.tryFind expression - |> Option.map (fun evidence -> - let targetType = representationType state evidence.TargetType - ({ loweredArgument with Type = targetType; Node = Hir.RecordConvert(loweredArgument, targetType) }: Hir.Expression)) - |> Option.defaultValue loweredArgument - Ok(make (Hir.Apply(loweredFn, argumentWithEvidence))) - | Error left, Error right -> Error(left @ right) - | Error errors, _ - | _, Error errors -> Error errors - | EIf(condition, whenTrue, whenFalse, _) -> - match lowerExpression state environment TBool condition, - lowerExpression state environment inferred whenTrue, - lowerExpression state environment inferred whenFalse with - | Ok c, Ok t, Ok f -> Ok(make (Hir.If(c, t, f))) - | results -> - let errors = - [ match results with - | Error e, _, _ -> yield! e - | _ -> () - match results with - | _, Error e, _ -> yield! e - | _ -> () - match results with - | _, _, Error e -> yield! e - | _ -> () ] - Error errors - | EBinOp(op, left, right, _) -> - let capturedOperandType = expressionType state left inferred - let operandType = - match capturedOperandType, inferred with - | TVar _, (TInt | TFloat as concrete) -> concrete - | value, _ -> value - if op = "@" then - match lowerExpression state environment inferred left, lowerExpression state environment inferred right with - | Ok l, Ok r -> Ok(make (Hir.Append(l, r))) - | Error a, Error b -> Error(a @ b) - | Error errors, _ | _, Error errors -> Error errors - else - match operation op operandType with - | None -> Error [ Diagnostic.hir "FSH1003" $"Unresolved primitive operation '{op}'." (Some span) ] - | Some primitive -> - match lowerExpression state environment operandType left, lowerExpression state environment operandType right with - | Ok l, Ok r -> Ok(make (Hir.Primitive(primitive, [ l; r ]))) - | Error l, Error r -> Error(l @ r) - | Error errors, _ - | _, Error errors -> Error errors - | ETuple(items, _) -> - let itemTypes = match inferred with TTuple values -> values | _ -> List.replicate items.Length (Types.freshVar()) - List.zip items itemTypes |> List.map (fun (item, itemType) -> lowerExpression state environment itemType item) - |> collectResults |> Result.map (Hir.Tuple >> make) - | EList(items, _) -> - let itemType = match inferred with TList value -> value | _ -> Types.freshVar() - items |> List.map (lowerExpression state environment itemType) |> collectResults |> Result.map (Hir.List >> make) - | ERecord(fields, _) | EStructuralRecord(fields, _) -> - let fieldTypes = recordFields state inferred - fields |> List.map (fun (name, item) -> lowerExpression state environment (fieldTypes |> Map.tryFind name |> Option.defaultValue (Types.freshVar())) item |> Result.map (fun value -> name, value)) - |> collectResults |> Result.map (Hir.Record >> make) - | EFieldGet(target, name, _) -> - match target with - | EVar(moduleName, _) -> - match environment |> Map.tryFind $"{moduleName}.{name}" with - | Some id -> Ok(make (Hir.Reference id)) - | None -> lowerExpression state environment (expressionType state target (TRecord Map.empty)) target |> Result.map (fun value -> make (Hir.FieldGet(value, name))) - | _ -> lowerExpression state environment (expressionType state target (TRecord Map.empty)) target |> Result.map (fun value -> make (Hir.FieldGet(value, name))) - | ESome(item, _) -> - let itemType = match inferred with TOption value -> value | _ -> Types.freshVar() - lowerExpression state environment itemType item |> Result.map (Hir.SomeValue >> make) - | ENone _ -> Ok(make Hir.NoneValue) - | ECons(head, tail, _) -> - let itemType = match inferred with TList value -> value | _ -> Types.freshVar() - match lowerExpression state environment itemType head, lowerExpression state environment inferred tail with - | Ok h, Ok t -> Ok(make (Hir.Cons(h, t))) | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e - | EAppend(left, right, _) -> - match lowerExpression state environment inferred left, lowerExpression state environment inferred right with - | Ok l, Ok r -> Ok(make (Hir.Append(l, r))) | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e - | ELet(name, binding, body, isRecursive, _, _) -> - let id = claimSymbol state TypeInfer.LocalSymbol name span - let bindingType = expressionType state binding (Types.freshVar()) - let bodyEnvironment = environment |> Map.add name id - let bindingEnvironment = if isRecursive then bodyEnvironment else environment - match lowerExpression state bindingEnvironment bindingType binding, lowerExpression state bodyEnvironment inferred body with - | Ok value, Ok loweredBody -> Ok(make (Hir.Let(id, name, isRecursive, value, loweredBody))) - | Error a, Error b -> Error(a @ b) - | Error errors, _ | _, Error errors -> Error errors - | ELetRecGroup(bindings, body, _) -> - let ids = bindings |> List.map (fun (name, _, _, _, bindingSpan) -> name, claimSymbol state TypeInfer.LocalSymbol name bindingSpan) |> Map.ofList - let recursiveEnvironment = Map.fold (fun env name id -> Map.add name id env) environment ids - let loweredBindings = - bindings - |> List.map (fun (name, parameters, _, valueExpression, bindingSpan) -> - let folded = Seq.foldBack (fun parameter inner -> ELambda(parameter, inner, bindingSpan)) parameters valueExpression - let bindingType = expressionType state folded (Types.freshVar()) - lowerExpression state recursiveEnvironment bindingType folded - |> Result.map (fun value -> ids.[name], name, bindingType, value)) - |> collectResults - match loweredBindings, lowerExpression state recursiveEnvironment inferred body with - | Ok values, Ok loweredBody -> Ok(make (Hir.LetRecursiveGroup(values, loweredBody))) - | Error a, Error b -> Error(a @ b) - | Error errors, _ | _, Error errors -> Error errors - | ERaise(value, _) -> lowerExpression state environment TString value |> Result.map (Hir.Fail >> make) - | ELetPattern(pattern, binding, body, _) -> - let bindingType = expressionType state binding (Types.freshVar()) - match lowerExpression state environment bindingType binding, lowerPattern state bindingType pattern with - | Ok loweredBinding, Ok(loweredPattern, bindings) -> - let bodyEnvironment = Map.fold (fun env name id -> Map.add name id env) environment bindings - lowerExpression state bodyEnvironment inferred body - |> Result.map (fun loweredBody -> make (Hir.Match(loweredBinding, [ { Pattern = loweredPattern; Guard = None; Body = loweredBody; Span = Ast.spanOfPattern pattern } ]))) - | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e - | EFor(name, source, body, _) -> - let sourceType = expressionType state source (TList(Types.freshVar())) - let itemType = match sourceType with TList item -> item | _ -> Types.freshVar() - let id = claimSymbol state TypeInfer.LocalSymbol name span - match lowerExpression state environment sourceType source, lowerExpression state (environment |> Map.add name id) TUnit body with - | Ok loweredSource, Ok loweredBody -> Ok(make (Hir.ForEach(id, name, itemType, loweredSource, loweredBody))) - | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e - | EMatch(target, cases, _) -> - let targetType = expressionType state target (Types.freshVar()) - match lowerExpression state environment targetType target with - | Error errors -> Error errors - | Ok loweredTarget -> - cases |> List.map (fun (pattern, guard, body, caseSpan) -> - lowerPattern state targetType pattern |> Result.bind (fun (loweredPattern, bindings) -> - let caseEnvironment = Map.fold (fun env name id -> Map.add name id env) environment bindings - match guard |> Option.map (fun value -> lowerExpression state caseEnvironment TBool value |> Result.map Some) |> Option.defaultValue (Ok None), lowerExpression state caseEnvironment inferred body with - | Ok loweredGuard, Ok loweredBody -> Ok({ Pattern = loweredPattern; Guard = loweredGuard; Body = loweredBody; Span = caseSpan }: Hir.MatchCase) - | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e)) - |> collectResults |> Result.map (fun loweredCases -> make (Hir.Match(loweredTarget, loweredCases))) - | ERange(startValue, endValue, _) -> - match lowerExpression state environment TInt startValue, lowerExpression state environment TInt endValue with - | Ok startItem, Ok endItem -> Ok(make (Hir.Range(startItem, endItem))) - | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e - | ERecordUpdate(target, fields, _) -> - let targetType = expressionType state target inferred - match lowerExpression state environment targetType target with - | Error errors -> Error errors - | Ok loweredTarget -> - let fieldTypes = recordFields state targetType - fields |> List.map (fun (name, item) -> lowerExpression state environment (fieldTypes |> Map.tryFind name |> Option.defaultValue (Types.freshVar())) item |> Result.map (fun value -> name, value)) - |> collectResults |> Result.map (fun loweredFields -> ({ Type = targetType; Span = span; Node = Hir.RecordUpdate(loweredTarget, loweredFields) }: Hir.Expression)) - | EStructuralRecordUpdate(target, fields, _) -> - match lowerExpression state environment inferred target with - | Error errors -> Error errors - | Ok loweredTarget -> - let fieldTypes = recordFields state inferred - fields |> List.map (fun (name, item) -> lowerExpression state environment (fieldTypes |> Map.tryFind name |> Option.defaultValue (Types.freshVar())) item |> Result.map (fun value -> name, value)) - |> collectResults |> Result.map (fun loweredFields -> make (Hir.RecordUpdate(loweredTarget, loweredFields))) - | EIndexGet(target, index, _) -> - let targetType = expressionType state target (Types.freshVar()) - let indexType = match targetType with TList _ -> TInt | TMap(key, _) -> key | _ -> Types.freshVar() - match lowerExpression state environment targetType target, lowerExpression state environment indexType index with - | Ok loweredTarget, Ok loweredIndex -> Ok(make (Hir.IndexGet(loweredTarget, loweredIndex))) - | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e - | ENameOf(name, _) -> Ok(make (Hir.Constant(Hir.String name))) - | ETypeOf(name, _) -> - match state.TypeDefinitions |> Map.tryFind name with - | Some value -> Ok(make (Hir.TypeToken value)) - | None -> Error [ Diagnostic.hir "FSH1005" $"Unknown type '{name}'." (Some span) ] - | EInterpolatedString(parts, _) -> - parts |> List.map (function - | IPText text -> Ok(Hir.TextPart text) - | IPExpr item -> lowerExpression state environment (expressionType state item (Types.freshVar())) item |> Result.map Hir.ValuePart) - |> collectResults |> Result.map (Hir.Interpolate >> make) - | EMap(entries, _) -> - let keyType, valueType = match inferred with TMap(key, value) -> key, value | _ -> Types.freshVar(), Types.freshVar() - entries |> List.map (function - | MEKeyValue(key, value) -> - match lowerExpression state environment keyType key, lowerExpression state environment valueType value with - | Ok loweredKey, Ok loweredValue -> Ok(Hir.KeyValue(loweredKey, loweredValue)) - | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e - | MESpread value -> lowerExpression state environment inferred value |> Result.map Hir.Spread) - |> collectResults |> Result.map (Hir.Map >> make) + let rec private lowerResolvedExpression state (expression: TypeInfer.ResolvedExpression) : Hir.Expression = + let expressionType = representationType state expression.Type + let make node = ({ Type = expressionType; Span = expression.Span; Node = node }: Hir.Expression) + let lower = lowerResolvedExpression state + match expression.Node with + | TypeInfer.ResolvedConstant value -> make (Hir.Constant(resolvedLiteral value)) + | TypeInfer.ResolvedReference reference -> make (Hir.Reference(Hir.SymbolId reference.SymbolId)) + | TypeInfer.ResolvedLambda(binding, body) -> + make (Hir.Lambda(Hir.SymbolId binding.SymbolId, binding.Name, representationType state binding.Type, lower body)) + | TypeInfer.ResolvedApply(fn, argument) -> + let loweredArgument = lower argument + let argumentWithEvidence = + expression.RecordCoercion + |> Option.map (fun evidence -> + let targetType = representationType state evidence.TargetType + ({ Type = targetType; Span = loweredArgument.Span; Node = Hir.RecordConvert(loweredArgument, targetType) }: Hir.Expression)) + |> Option.defaultValue loweredArgument + make (Hir.Apply(lower fn, argumentWithEvidence)) + | TypeInfer.ResolvedIf(condition, whenTrue, whenFalse) -> make (Hir.If(lower condition, lower whenTrue, lower whenFalse)) + | TypeInfer.ResolvedPrimitive(operation, arguments) -> make (Hir.Primitive(resolvedOperation operation, List.map lower arguments)) + | TypeInfer.ResolvedTuple items -> make (Hir.Tuple(List.map lower items)) + | TypeInfer.ResolvedList items -> make (Hir.List(List.map lower items)) + | TypeInfer.ResolvedRecord fields -> make (Hir.Record(fields |> List.map (fun (name, value) -> name, lower value))) + | TypeInfer.ResolvedFieldGet(target, name) -> make (Hir.FieldGet(lower target, name)) + | TypeInfer.ResolvedSome item -> make (Hir.SomeValue(lower item)) + | TypeInfer.ResolvedNone -> make Hir.NoneValue + | TypeInfer.ResolvedCons(head, tail) -> make (Hir.Cons(lower head, lower tail)) + | TypeInfer.ResolvedAppend(left, right) -> make (Hir.Append(lower left, lower right)) + | TypeInfer.ResolvedLet(binding, recursive, value, body) -> + make (Hir.Let(Hir.SymbolId binding.SymbolId, binding.Name, recursive, lower value, lower body)) + | TypeInfer.ResolvedLetRecursiveGroup(bindings, body) -> + let values = bindings |> List.map (fun binding -> Hir.SymbolId binding.Binding.SymbolId, binding.Binding.Name, representationType state binding.Binding.Type, lower binding.Value) + make (Hir.LetRecursiveGroup(values, lower body)) + | TypeInfer.ResolvedFail value -> make (Hir.Fail(lower value)) + | TypeInfer.ResolvedMatch(target, cases) -> + let loweredCases = + cases |> List.map (fun case -> + let pattern, _ = lowerResolvedPattern state case.Pattern + ({ Pattern = pattern; Guard = Option.map lower case.Guard; Body = lower case.Body; Span = case.Span }: Hir.MatchCase)) + make (Hir.Match(lower target, loweredCases)) + | TypeInfer.ResolvedRange(first, last) -> make (Hir.Range(lower first, lower last)) + | TypeInfer.ResolvedRecordUpdate(preserveTargetRepresentation, target, fields) -> + let loweredTarget = lower target + let resultType = if preserveTargetRepresentation then loweredTarget.Type else expressionType + ({ Type = resultType + Span = expression.Span + Node = Hir.RecordUpdate(loweredTarget, fields |> List.map (fun (name, value) -> name, lower value)) }: Hir.Expression) + | TypeInfer.ResolvedIndexGet(target, index) -> make (Hir.IndexGet(lower target, lower index)) + | TypeInfer.ResolvedInterpolate parts -> + parts |> List.map (function TypeInfer.ResolvedTextPart text -> Hir.TextPart text | TypeInfer.ResolvedValuePart value -> Hir.ValuePart(lower value)) |> Hir.Interpolate |> make + | TypeInfer.ResolvedMap entries -> + entries |> List.map (function TypeInfer.ResolvedKeyValue(key, value) -> Hir.KeyValue(lower key, lower value) | TypeInfer.ResolvedSpread value -> Hir.Spread(lower value)) |> Hir.Map |> make + | TypeInfer.ResolvedForEach(binding, source, body) -> + make (Hir.ForEach(Hir.SymbolId binding.SymbolId, binding.Name, representationType state binding.Type, lower source, lower body)) + | TypeInfer.ResolvedTypeToken tokenType -> make (Hir.TypeToken(representationType state tokenType)) let lower (externs: CompilerExtern list) (resolved: TypeInfer.ResolvedProgram) : Result = let exportMetadata name span enabled = if enabled then Some({ Name = name; Span = span }: Hir.ExportMetadata) else None - let attributesFor statement = - resolved.ResolvedStatements - |> List.tryFind (fun resolvedStatement -> resolvedStatement.Source = statement) - |> Option.map _.Attributes - |> Option.defaultValue [] + let attributesOf (statement: TypeInfer.ResolvedStatement) = + statement.Attributes |> List.map (fun attribute -> ({ Name = attribute.Name; Arguments = attribute.Arguments; Span = attribute.Span }: Hir.AttributeMetadata)) - let expressionTypes = - let rec addExpression state (item: TypeInfer.ResolvedExpression) = - item.Children |> List.fold addExpression (Map.add item.Source item.Type state) - resolved.ResolvedStatements - |> List.collect _.Expressions - |> List.fold addExpression Map.empty - let coercions = - let rec collect state (item: TypeInfer.ResolvedExpression) = - let state = item.RecordCoercion |> Option.map (fun evidence -> Map.add item.Source evidence state) |> Option.defaultValue state - item.Children |> List.fold collect state - resolved.ResolvedStatements |> List.collect _.Expressions |> List.fold collect Map.empty let nextSynthetic = resolved.Symbols |> List.map _.SymbolId |> function [] -> 0 | values -> List.max values + 1 let state = - { NextSymbol = nextSynthetic; ExpressionTypes = expressionTypes; Coercions = coercions - UnionCases = Map.empty; TypeDefinitions = Map.empty; AvailableSymbols = resolved.Symbols; Diagnostics = [] } + { NextSymbol = nextSynthetic + AvailableSymbols = resolved.Symbols + Diagnostics = [] } let resolvedSchemes = resolved.Symbols |> List.choose (fun symbol -> symbol.Scheme |> Option.map (fun scheme -> Hir.SymbolId symbol.SymbolId, scheme)) @@ -396,51 +188,24 @@ module HirLowering = |> Map.ofList let externSymbols = externs |> List.sortBy _.Name |> List.map (fun value -> value.Name, findSymbol state TypeInfer.ExternSymbol value.Name syntheticSpan) |> Map.ofList let typeDefinitions = - resolved.TypedProgram |> List.choose (function TypeInfer.TSType definition -> Some definition | _ -> None) - let rec typeFromRef = function - | TRName "unit" -> TUnit | TRName "int" -> TInt | TRName "float" -> TFloat | TRName "bool" -> TBool | TRName "string" -> TString - | TRName name -> TNamed name - | TRTuple items -> TTuple(List.map typeFromRef items) - | TRFun(argument, result) -> TFun(typeFromRef argument, typeFromRef result) - | TRPostfix(item, "list") -> TList(typeFromRef item) - | TRPostfix(item, "option") -> TOption(typeFromRef item) - | TRPostfix(item, "task") -> TTask(typeFromRef item) - | TRRecord fields | TRStructuralRecord fields -> TRecord(fields |> List.map (fun (name, item) -> name, typeFromRef item) |> Map.ofList) - | value -> TNamed(sprintf "%A" value) + resolved.ResolvedStatements + |> List.choose (fun statement -> + match statement.Node with + | TypeInfer.ResolvedTypeDeclaration definition -> Some(statement, definition) + | _ -> None) let unionDefinitions = - typeDefinitions |> List.filter (fun definition -> not definition.Cases.IsEmpty) |> List.map (fun definition -> - let cases = definition.Cases |> List.map (fun (name, payload) -> name, Option.map typeFromRef payload) |> Map.ofList - definition, TUnion(definition.Name, cases)) - state.TypeDefinitions <- - typeDefinitions |> List.map (fun definition -> - let value = - if not definition.Cases.IsEmpty then - TUnion(definition.Name, definition.Cases |> List.map (fun (name, payload) -> name, Option.map typeFromRef payload) |> Map.ofList) - else TRecord(definition.Fields |> List.map (fun (name, fieldType) -> name, typeFromRef fieldType) |> Map.ofList) - definition.Name, value) |> Map.ofList - state.UnionCases <- - unionDefinitions |> List.collect (fun (definition, _) -> - let cases = definition.Cases |> List.map (fun (name, payload) -> name, Option.map typeFromRef payload) |> Map.ofList - definition.Cases |> List.map (fun (caseName, _) -> caseName, (definition.Name, cases.[caseName]))) |> Map.ofList - let typeSymbols = typeDefinitions - |> List.map (fun definition -> definition.Name, findSymbol state TypeInfer.TypeSymbol definition.Name definition.Span) - |> Map.ofList + |> List.choose (fun (statement, definition) -> + match definition.Definition with + | TUnion(name, cases) -> Some(statement, definition, name, cases) + | _ -> None) let constructorSymbols = - unionDefinitions |> List.collect (fun (definition, _) -> definition.Cases |> List.map (fun (caseName, _) -> definition.Name, caseName, findSymbol state TypeInfer.UnionCaseSymbol $"{definition.Name}.{caseName}" definition.Span)) - let topLevel = - resolved.TypedProgram - |> List.collect (function - | TypeInfer.TSLet(name, _, _, _, _, span) -> [ name, span ] - | TypeInfer.TSLetPattern(_, _, bindings, _, span) -> bindings |> Map.keys |> Seq.map (fun name -> name, span) |> Seq.toList - | TypeInfer.TSLetRecGroup(bindings, _, _) -> bindings |> List.map (fun (name, _, _, span) -> name, span) - | TypeInfer.TSBackendImport(name, _, _, span) -> [ name, span ] - | _ -> []) - |> List.distinct - |> List.fold (fun env (name, span) -> env |> Map.add name (findSymbol state TypeInfer.DeclarationSymbol name span)) (Map.fold (fun env name id -> Map.add name id env) builtins externSymbols) - let topLevel = - constructorSymbols |> List.fold (fun env (typeName, caseName, id) -> env |> Map.add caseName id |> Map.add $"{typeName}.{caseName}" id) topLevel - + unionDefinitions + |> List.collect (fun (_, definition, name, cases) -> + cases + |> Map.toList + |> List.map (fun (caseName, _) -> + name, caseName, findSymbol state TypeInfer.UnionCaseSymbol $"{name}.{caseName}" definition.Binding.Span)) let mutable declarations = let builtinDeclarations = BuiltinSignatures.builtinSchemes |> Map.toList |> List.sortBy fst |> List.map (fun (name, scheme) -> @@ -458,132 +223,131 @@ module HirLowering = | UnsupportedHostExtern -> Hir.HostExternal(external.Name, external.Arity) ({ Id = externSymbols.[external.Name]; Name = external.Name; Scheme = schemeFor externSymbols.[external.Name] external.Scheme; Attributes = []; IsRecursive = false; IsExported = false; Export = None; Span = span; Value = { Type = valueType; Span = span; Node = node } }: Hir.Declaration)) builtinDeclarations @ externalDeclarations @ (constructorSymbols |> List.map (fun (typeName, caseName, id) -> - let definition, _ = unionDefinitions |> List.find (fun (definition, _) -> definition.Name = typeName) + let _, definition, _, cases = unionDefinitions |> List.find (fun (_, _, name, _) -> name = typeName) let unionType = TNamed typeName - let payloadType = definition.Cases |> List.find (fst >> (=) caseName) |> snd |> Option.map typeFromRef + let payloadType = cases.[caseName] + let span = definition.Binding.Span let value = match payloadType with - | None -> ({ Type = unionType; Span = definition.Span; Node = Hir.UnionCase(typeName, caseName, None) }: Hir.Expression) + | None -> ({ Type = unionType; Span = span; Node = Hir.UnionCase(typeName, caseName, None) }: Hir.Expression) | Some itemType -> let parameter = newSymbol state - let body = ({ Type = unionType; Span = definition.Span; Node = Hir.UnionCase(typeName, caseName, Some({ Type = itemType; Span = definition.Span; Node = Hir.Reference parameter })) }: Hir.Expression) - ({ Type = TFun(itemType, unionType); Span = definition.Span; Node = Hir.Lambda(parameter, "value", itemType, body) }: Hir.Expression) - ({ Id = id; Name = $"{typeName}.{caseName}"; Scheme = Types.generalize Map.empty value.Type; Attributes = []; IsRecursive = false; IsExported = false; Export = None; Span = definition.Span; Value = value }: Hir.Declaration))) + let body = ({ Type = unionType; Span = span; Node = Hir.UnionCase(typeName, caseName, Some({ Type = itemType; Span = span; Node = Hir.Reference parameter })) }: Hir.Expression) + ({ Type = TFun(itemType, unionType); Span = span; Node = Hir.Lambda(parameter, "value", itemType, body) }: Hir.Expression) + ({ Id = id; Name = $"{typeName}.{caseName}"; Scheme = Types.generalize Map.empty value.Type; Attributes = []; IsRecursive = false; IsExported = false; Export = None; Span = span; Value = value }: Hir.Declaration))) let mutable entry: Hir.Expression option = None let mutable errors = [] - for statement in resolved.TypedProgram do - match statement with - | TypeInfer.TSLet(name, expression, inferred, isRecursive, isExported, span) -> - let statement = TypeInfer.TSLet(name, expression, inferred, isRecursive, isExported, span) - match lowerExpression state topLevel inferred expression with - | Ok value -> - declarations <- declarations @ - [ ({ Id = Map.find name topLevel; - Name = name; - Scheme = schemeFor (Map.find name topLevel) (Types.generalize Map.empty inferred); - Attributes = attributesFor statement; - IsRecursive = isRecursive; - IsExported = isExported; - Export = (exportMetadata name span isExported); - Span = span; - Value = value }: Hir.Declaration) ] - | Error diagnostics -> errors <- errors @ diagnostics - | TypeInfer.TSExpr expression -> - match lowerExpression state topLevel expression.Type expression.Expr with - | Ok value -> - let id = newSymbol state - let name = $"$expression{Hir.symbolNumber id}" - declarations <- declarations @ - [ ({ Id = id - Name = name - Scheme = Types.generalize Map.empty expression.Type - Attributes = [] - IsRecursive = false - IsExported = false - Export = None - Span = value.Span - Value = value }: Hir.Declaration) ] - entry <- Some { Type = expression.Type; Span = value.Span; Node = Hir.Reference id } - | Error diagnostics -> errors <- errors @ diagnostics - | TypeInfer.TSLetPattern(pattern, expression, bindings, isExported, span) -> - let statement = TypeInfer.TSLetPattern(pattern, expression, bindings, isExported, span) - let bindingType = expressionType state expression (Types.freshVar()) - match lowerExpression state topLevel bindingType expression, lowerPattern state bindingType pattern with - | Ok value, Ok(loweredPattern, patternBindings) -> - let temporaryId = newSymbol state - declarations <- declarations @ - [ ({ Id = temporaryId - Name = $"$pattern{Hir.symbolNumber temporaryId}" - Scheme = Types.generalize Map.empty bindingType - Attributes = [] - IsRecursive = false - IsExported = false - Export = None - Span = span - Value = value }: Hir.Declaration) ] - let target = ({ Type = bindingType; Span = span; Node = Hir.Reference temporaryId }: Hir.Expression) - let unitBody = ({ Type = TUnit; Span = span; Node = Hir.Constant Hir.Unit }: Hir.Expression) - let check = ({ Type = TUnit; Span = span; Node = Hir.Match(target, [ { Pattern = loweredPattern; Guard = None; Body = unitBody; Span = span } ]) }: Hir.Expression) - let checkId = newSymbol state - declarations <- declarations @ - [ ({ Id = checkId; Name = $"$patternCheck{Hir.symbolNumber checkId}"; Scheme = Forall([], TUnit); Attributes = []; IsRecursive = false; IsExported = false; Export = None; Span = span; Value = check }: Hir.Declaration) ] - for name, inferred in bindings |> Map.toList do - match patternBindings |> Map.tryFind name with - | None -> errors <- errors @ [ Diagnostic.hir "FSH1006" $"Pattern binding '{name}' was not resolved." (Some span) ] - | Some patternId -> - let body = ({ Type = inferred; Span = span; Node = Hir.Reference patternId }: Hir.Expression) - let projection = ({ Type = inferred; Span = span; Node = Hir.Match(target, [ { Pattern = loweredPattern; Guard = None; Body = body; Span = span } ]) }: Hir.Expression) - declarations <- declarations @ - [ ({ Id = Map.find name topLevel; Name = name; Scheme = schemeFor (Map.find name topLevel) (Types.generalize Map.empty inferred); Attributes = attributesFor statement; IsRecursive = false; IsExported = isExported; Export = (exportMetadata name span isExported); Span = span; Value = projection }: Hir.Declaration) ] - | Error a, Error b -> errors <- errors @ a @ b - | Error diagnostics, _ | _, Error diagnostics -> errors <- errors @ diagnostics - | TypeInfer.TSLetRecGroup(bindings, isExported, groupSpan) -> - let statement = TypeInfer.TSLetRecGroup(bindings, isExported, groupSpan) - for name, expression, inferred, span in bindings do - match lowerExpression state topLevel inferred expression with - | Ok value -> - declarations <- declarations @ - [ ({ Id = Map.find name topLevel - Name = name - Scheme = schemeFor (Map.find name topLevel) (Types.generalize Map.empty inferred) - Attributes = attributesFor statement - IsRecursive = true - IsExported = isExported - Export = (exportMetadata name span isExported) - Span = span - Value = value }: Hir.Declaration) ] - | Error diagnostics -> errors <- errors @ diagnostics - | TypeInfer.TSBackendImport(name, importedType, import, span) -> - let statement = TypeInfer.TSBackendImport(name, importedType, import, span) + for resolvedStatement in resolved.ResolvedStatements do + let isExported = resolvedStatement.Export.IsSome + match resolvedStatement.Node with + | TypeInfer.ResolvedValueDeclaration declaration -> + let binding = declaration.Binding + let value = lowerResolvedExpression state declaration.Value declarations <- declarations @ - [ ({ Id = Map.find name topLevel + [ ({ Id = Hir.SymbolId binding.SymbolId + Name = binding.Name + Scheme = declaration.Scheme + Attributes = attributesOf resolvedStatement + IsRecursive = declaration.IsRecursive + IsExported = isExported + Export = exportMetadata binding.Name binding.Span isExported + Span = binding.Span + Value = value }: Hir.Declaration) ] + | TypeInfer.ResolvedExpressionStatement expression -> + let value = lowerResolvedExpression state expression + let id = newSymbol state + let name = $"$expression{Hir.symbolNumber id}" + declarations <- declarations @ + [ ({ Id = id Name = name - Scheme = schemeFor (Map.find name topLevel) (Forall([], importedType)) - Attributes = attributesFor statement + Scheme = Types.generalize Map.empty value.Type + Attributes = [] + IsRecursive = false + IsExported = false + Export = None + Span = value.Span + Value = value }: Hir.Declaration) ] + entry <- Some { Type = value.Type; Span = value.Span; Node = Hir.Reference id } + | TypeInfer.ResolvedPatternDeclaration declaration -> + let span = resolvedStatement.Span + let value = lowerResolvedExpression state declaration.Value + let loweredPattern, patternBindings = lowerResolvedPattern state declaration.Pattern + let bindingType = value.Type + let temporaryId = newSymbol state + declarations <- declarations @ + [ ({ Id = temporaryId + Name = $"$pattern{Hir.symbolNumber temporaryId}" + Scheme = Types.generalize Map.empty bindingType + Attributes = [] IsRecursive = false IsExported = false Export = None Span = span - Value = { Type = importedType; Span = span; Node = Hir.TargetImport(import.ModulePath, import.MemberName) } }: Hir.Declaration) ] - | TypeInfer.TSType _ -> () + Value = value }: Hir.Declaration) ] + let target = ({ Type = bindingType; Span = span; Node = Hir.Reference temporaryId }: Hir.Expression) + let unitBody = ({ Type = TUnit; Span = span; Node = Hir.Constant Hir.Unit }: Hir.Expression) + let check = ({ Type = TUnit; Span = span; Node = Hir.Match(target, [ { Pattern = loweredPattern; Guard = None; Body = unitBody; Span = span } ]) }: Hir.Expression) + let checkId = newSymbol state + declarations <- declarations @ + [ ({ Id = checkId; Name = $"$patternCheck{Hir.symbolNumber checkId}"; Scheme = Forall([], TUnit); Attributes = []; IsRecursive = false; IsExported = false; Export = None; Span = span; Value = check }: Hir.Declaration) ] + for binding, scheme in declaration.Bindings do + let name = binding.Name + let inferred = representationType state binding.Type + match patternBindings |> Map.tryFind name with + | None -> errors <- errors @ [ Diagnostic.hir "FSH1006" $"Pattern binding '{name}' was not resolved." (Some span) ] + | Some patternId -> + let body = ({ Type = inferred; Span = span; Node = Hir.Reference patternId }: Hir.Expression) + let projection = ({ Type = inferred; Span = span; Node = Hir.Match(target, [ { Pattern = loweredPattern; Guard = None; Body = body; Span = span } ]) }: Hir.Expression) + declarations <- declarations @ + [ ({ Id = Hir.SymbolId binding.SymbolId; Name = name; Scheme = scheme; Attributes = attributesOf resolvedStatement; IsRecursive = false; IsExported = isExported; Export = (exportMetadata name span isExported); Span = span; Value = projection }: Hir.Declaration) ] + | TypeInfer.ResolvedRecursiveDeclarations recursiveDeclarations -> + for declaration in recursiveDeclarations do + let binding = declaration.Binding + let value = lowerResolvedExpression state declaration.Value + declarations <- declarations @ + [ ({ Id = Hir.SymbolId binding.SymbolId + Name = binding.Name + Scheme = declaration.Scheme + Attributes = attributesOf resolvedStatement + IsRecursive = true + IsExported = isExported + Export = exportMetadata binding.Name binding.Span isExported + Span = binding.Span + Value = value }: Hir.Declaration) ] + | TypeInfer.ResolvedImportedDeclaration declaration -> + let binding = declaration.Binding + declarations <- declarations @ + [ ({ Id = Hir.SymbolId binding.SymbolId + Name = binding.Name + Scheme = declaration.Scheme + Attributes = attributesOf resolvedStatement + IsRecursive = false + IsExported = false + Export = None + Span = binding.Span + Value = { Type = representationType state binding.Type; Span = binding.Span; Node = Hir.TargetImport(declaration.Import.ModulePath, declaration.Import.MemberName) } }: Hir.Declaration) ] + | TypeInfer.ResolvedTypeDeclaration _ -> () let importedRecords = typeDefinitions - |> List.choose (fun definition -> + |> List.choose (fun (_, definition) -> definition.Import |> Option.map (fun import -> - ({ Name = definition.Name; ModulePath = import.ModulePath; MemberName = import.MemberName; IsUnion = not definition.Cases.IsEmpty - Fields = definition.Fields |> List.map (fun (name, fieldType) -> name, typeFromRef fieldType) - Span = definition.Span }: Hir.ImportedRecord))) + ({ Name = definition.Binding.Name + ModulePath = import.ModulePath + MemberName = import.MemberName + IsUnion = match definition.Definition with TUnion _ -> true | _ -> false + Fields = definition.Fields + Span = definition.Binding.Span }: Hir.ImportedRecord))) let hirTypes = typeDefinitions - |> List.map (fun definition -> - ({ Id = typeSymbols.[definition.Name] - Name = definition.Name + |> List.map (fun (statement, definition) -> + ({ Id = Hir.SymbolId definition.Binding.SymbolId + Name = definition.Binding.Name IsRecursive = definition.IsRecursive - Definition = state.TypeDefinitions.[definition.Name] - Attributes = attributesFor (TypeInfer.TSType definition) - Import = importedRecords |> List.tryFind (fun imported -> imported.Name = definition.Name) - Span = definition.Span }: Hir.TypeDeclaration)) + Definition = definition.Definition + Attributes = attributesOf statement + Import = importedRecords |> List.tryFind (fun imported -> imported.Name = definition.Binding.Name) + Span = definition.Binding.Span }: Hir.TypeDeclaration)) let diagnostics = errors @ state.Diagnostics if diagnostics.IsEmpty then Ok { Types = hirTypes; ImportedRecords = importedRecords; Declarations = declarations; Entry = entry } else Error diagnostics diff --git a/src/FScript.Language/TypeInfer.fs b/src/FScript.Language/TypeInfer.fs index 077ce98..2816495 100644 --- a/src/FScript.Language/TypeInfer.fs +++ b/src/FScript.Language/TypeInfer.fs @@ -167,10 +167,11 @@ module TypeInfer = type ResolvedSymbolReference = { SymbolId: int Name: string + Kind: ResolvedSymbolKind Type: Type Span: Span } - type ResolvedSymbolKind = + and ResolvedSymbolKind = | TypeSymbol | DeclarationSymbol | ParameterSymbol @@ -201,15 +202,92 @@ module TypeInfer = TargetType: Type TargetName: string option } - type ResolvedExpression = - { Source: Expr + type ResolvedLiteral = + | ResolvedUnit + | ResolvedInt of int64 + | ResolvedFloat of float + | ResolvedBool of bool + | ResolvedString of string + + type ResolvedBinding = + { SymbolId: int + Name: string Type: Type + Span: Span } + + type ResolvedExpression = + { Type: Type Span: Span Reference: ResolvedSymbolReference option Primitive: ResolvedPrimitiveOperation option RecordCoercion: RecordCoercionEvidence option + Node: ResolvedExpressionNode Children: ResolvedExpression list } + and ResolvedExpressionNode = + | ResolvedConstant of ResolvedLiteral + | ResolvedReference of ResolvedSymbolReference + | ResolvedLambda of ResolvedBinding * ResolvedExpression + | ResolvedApply of ResolvedExpression * ResolvedExpression + | ResolvedIf of ResolvedExpression * ResolvedExpression * ResolvedExpression + | ResolvedPrimitive of ResolvedPrimitiveOperation * ResolvedExpression list + | ResolvedTuple of ResolvedExpression list + | ResolvedList of ResolvedExpression list + | ResolvedRecord of (string * ResolvedExpression) list + | ResolvedFieldGet of ResolvedExpression * string + | ResolvedSome of ResolvedExpression + | ResolvedNone + | ResolvedCons of ResolvedExpression * ResolvedExpression + | ResolvedAppend of ResolvedExpression * ResolvedExpression + | ResolvedLet of ResolvedBinding * bool * ResolvedExpression * ResolvedExpression + | ResolvedLetRecursiveGroup of ResolvedRecursiveBinding list * ResolvedExpression + | ResolvedFail of ResolvedExpression + | ResolvedMatch of ResolvedExpression * ResolvedMatchCase list + | ResolvedRange of ResolvedExpression * ResolvedExpression + | ResolvedRecordUpdate of bool * ResolvedExpression * (string * ResolvedExpression) list + | ResolvedIndexGet of ResolvedExpression * ResolvedExpression + | ResolvedInterpolate of ResolvedInterpolationPart list + | ResolvedMap of ResolvedMapEntry list + | ResolvedForEach of ResolvedBinding * ResolvedExpression * ResolvedExpression + | ResolvedTypeToken of Type + + and ResolvedRecursiveBinding = + { Binding: ResolvedBinding + Value: ResolvedExpression } + + and ResolvedInterpolationPart = + | ResolvedTextPart of string + | ResolvedValuePart of ResolvedExpression + + and ResolvedMapEntry = + | ResolvedKeyValue of ResolvedExpression * ResolvedExpression + | ResolvedSpread of ResolvedExpression + + and ResolvedPattern = + { Type: Type + Span: Span + Node: ResolvedPatternNode } + + and ResolvedPatternNode = + | ResolvedWildcard + | ResolvedPatternBinding of ResolvedBinding + | ResolvedLiteralPattern of ResolvedLiteral + | ResolvedEmptyList + | ResolvedConsPattern of ResolvedPattern * ResolvedPattern + | ResolvedTuplePattern of ResolvedPattern list + | ResolvedRecordPattern of (string * ResolvedPattern) list + | ResolvedMapPattern of (ResolvedPattern * ResolvedPattern) list * ResolvedPattern option + | ResolvedSomePattern of ResolvedPattern + | ResolvedNonePattern + | ResolvedUnionPattern of string * string * ResolvedPattern option + | ResolvedTypePattern of Type + + and ResolvedMatchCase = + { Pattern: ResolvedPattern + Guard: ResolvedExpression option + Body: ResolvedExpression + Span: Span } + type private LocalVariableTypeCapture = { Name: string Span: Span @@ -223,10 +301,49 @@ module TypeInfer = | TSExpr of TypedExpr type TypedProgram = TypedStmt list + type ResolvedBackendImport = + { ModulePath: string + MemberName: string + Span: Span } + + type ResolvedTypeDeclaration = + { Binding: ResolvedBinding + IsRecursive: bool + Definition: Type + Fields: (string * Type) list + Cases: (string * Type option) list + Import: ResolvedBackendImport option } + + type ResolvedValueDeclaration = + { Binding: ResolvedBinding + Scheme: Scheme + IsRecursive: bool + Value: ResolvedExpression } + + type ResolvedPatternDeclaration = + { Pattern: ResolvedPattern + Value: ResolvedExpression + Bindings: (ResolvedBinding * Scheme) list } + + type ResolvedImportedDeclaration = + { Binding: ResolvedBinding + Scheme: Scheme + Import: ResolvedBackendImport } + + type ResolvedStatementNode = + | ResolvedTypeDeclaration of ResolvedTypeDeclaration + | ResolvedValueDeclaration of ResolvedValueDeclaration + | ResolvedPatternDeclaration of ResolvedPatternDeclaration + | ResolvedRecursiveDeclarations of ResolvedValueDeclaration list + | ResolvedImportedDeclaration of ResolvedImportedDeclaration + | ResolvedExpressionStatement of ResolvedExpression + type ResolvedStatement = - { Source: TypedStmt + { Span: Span Attributes: Attribute list Export: ResolvedExport option + Node: ResolvedStatementNode + Patterns: ResolvedPattern list Expressions: ResolvedExpression list } and ResolvedExport = @@ -1510,6 +1627,7 @@ module TypeInfer = let mutable nextSymbol = 0 let symbols = ResizeArray() let references = ResizeArray() + let patternBindingIds = ResizeArray() let normalizeScheme (Forall(variables, schemeType)) = let mapping = variables |> List.mapi (fun index variable -> variable, index) |> Map.ofList let rec normalize = function @@ -1573,19 +1691,41 @@ module TypeInfer = | TSExpr _ -> () let recordReference environment name expression span = environment |> Map.tryFind name |> Option.iter (fun id -> - references.Add { SymbolId = id; Name = name; Type = expressionTypes |> Map.tryFind expression |> Option.defaultValue TUnit |> normalizeTypeVariables; Span = span }) - let rec bindPattern kind environment pattern = - match pattern with - | PVar(name, span) -> Map.add name (allocate name kind span None) environment - | PCons(head, tail, _) -> bindPattern kind (bindPattern kind environment head) tail - | PTuple(items, _) -> items |> List.fold (bindPattern kind) environment - | PRecord(fields, _) -> fields |> List.fold (fun env (_, item) -> bindPattern kind env item) environment - | PMap(clauses, tail, _) -> - let withClauses = clauses |> List.fold (fun env (key, value) -> bindPattern kind (bindPattern kind env key) value) environment - tail |> Option.map (bindPattern kind withClauses) |> Option.defaultValue withClauses - | PSome(item, _) -> bindPattern kind environment item - | PUnionCase(_, _, payload, _) -> payload |> Option.map (bindPattern kind environment) |> Option.defaultValue environment - | _ -> environment + let symbol = symbols |> Seq.find (fun symbol -> symbol.SymbolId = id) + references.Add + { SymbolId = id + Name = name + Kind = symbol.Kind + Type = expressionTypes |> Map.tryFind expression |> Option.defaultValue TUnit |> normalizeTypeVariables + Span = span }) + let bindPattern kind environment pattern = + let rec bind bindings current pattern = + match pattern with + | PVar(name, span) -> + match Map.tryFind name bindings with + | Some id -> + patternBindingIds.Add(span, id) + bindings, Map.add name id current + | None -> + let id = allocate name kind span None + patternBindingIds.Add(span, id) + Map.add name id bindings, Map.add name id current + | PCons(head, tail, _) -> + let bindings, current = bind bindings current head + bind bindings current tail + | PTuple(items, _) -> items |> List.fold (fun (bindings, current) item -> bind bindings current item) (bindings, current) + | PRecord(fields, _) -> fields |> List.fold (fun (bindings, current) (_, item) -> bind bindings current item) (bindings, current) + | PMap(clauses, tail, _) -> + let bindings, current = + clauses + |> List.fold (fun (bindings, current) (key, value) -> + let bindings, current = bind bindings current key + bind bindings current value) (bindings, current) + tail |> Option.map (bind bindings current) |> Option.defaultValue (bindings, current) + | PSome(item, _) -> bind bindings current item + | PUnionCase(_, _, payload, _) -> payload |> Option.map (bind bindings current) |> Option.defaultValue (bindings, current) + | _ -> bindings, current + bind Map.empty environment pattern |> snd let rec visit environment expression = let visitSame item = visit environment item match expression with @@ -1638,33 +1778,97 @@ module TypeInfer = | TSType _ -> () let resolvedReferences = references |> Seq.toList let finalType expression = expressionTypes |> Map.tryFind expression |> Option.defaultValue TUnit |> normalizeTypeVariables - let rec resolveExpression expression = + let declarations = + (BuiltinTypes.typeDefs |> List.map (fun definition -> definition.Name, definition)) + @ (typed |> List.choose (function TSType definition -> Some(definition.Name, definition) | _ -> None)) + |> Map.ofList + let resolvedTypeDefinitions = + declarations + |> Map.map (fun name definition -> + if definition.Cases.IsEmpty then + definition.Fields + |> List.map (fun (fieldName, fieldType) -> fieldName, typeFromRef declarations [ name ] fieldType) + |> Map.ofList + |> TRecord + else + definition.Cases + |> List.map (fun (caseName, payload) -> caseName, Option.map (typeFromRef declarations [ name ]) payload) + |> Map.ofList + |> fun cases -> TUnion(name, cases)) + let resolvedLiteral = function + | LInt value -> ResolvedInt value + | LFloat value -> ResolvedFloat value + | LBool value -> ResolvedBool value + | LString value -> ResolvedString value + let symbolAt kind name span = + symbols + |> Seq.find (fun symbol -> symbol.Kind = kind && symbol.Name = name && symbol.Span = span) + let bindingAt kind name span bindingType = + let id = + if kind = PatternBindingSymbol then + patternBindingIds + |> Seq.tryFind (fst >> (=) span) + |> Option.map snd + |> Option.defaultWith (fun () -> (symbolAt kind name span).SymbolId) + else + (symbolAt kind name span).SymbolId + { SymbolId = id; Name = name; Type = bindingType; Span = span } + let referenceAt expression = + let span = Ast.spanOfExpr expression + resolvedReferences |> List.tryFind (fun item -> item.Span = span) + let rec resolvePattern (expected: Type) (pattern: Pattern) : ResolvedPattern = + let span = Ast.spanOfPattern pattern + let make node : ResolvedPattern = { Type = expected; Span = span; Node = node } + match pattern with + | PWildcard _ -> make ResolvedWildcard + | PVar(name, _) -> make (ResolvedPatternBinding(bindingAt PatternBindingSymbol name span expected)) + | PLiteral(value, _) -> make (ResolvedLiteralPattern(resolvedLiteral value)) + | PNil _ -> make ResolvedEmptyList + | PCons(head, tail, _) -> + let itemType = match expected with TList item -> item | _ -> Types.freshVar() + make (ResolvedConsPattern(resolvePattern itemType head, resolvePattern expected tail)) + | PTuple(items, _) -> + let itemTypes = match expected with TTuple types when types.Length = items.Length -> types | _ -> List.replicate items.Length (Types.freshVar()) + make (ResolvedTuplePattern(List.map2 resolvePattern itemTypes items)) + | PRecord(fields, _) -> + let fieldTypes = + match expected with + | TRecord values -> values + | TNamed name -> match Map.tryFind name resolvedTypeDefinitions with Some(TRecord values) -> values | _ -> Map.empty + | _ -> Map.empty + fields + |> List.map (fun (name, item) -> name, resolvePattern (fieldTypes |> Map.tryFind name |> Option.defaultValue (Types.freshVar())) item) + |> ResolvedRecordPattern + |> make + | PMap(clauses, tail, _) -> + let keyType, valueType = match expected with TMap(key, value) -> key, value | _ -> Types.freshVar(), Types.freshVar() + let resolvedClauses = clauses |> List.map (fun (key, value) -> resolvePattern keyType key, resolvePattern valueType value) + make (ResolvedMapPattern(resolvedClauses, Option.map (resolvePattern expected) tail)) + | PSome(item, _) -> + let itemType = match expected with TOption value -> value | _ -> Types.freshVar() + make (ResolvedSomePattern(resolvePattern itemType item)) + | PNone _ -> make ResolvedNonePattern + | PUnionCase(qualifier, caseName, payload, _) -> + let unionName, payloadType = + match expected with + | TNamed name -> + match Map.tryFind name resolvedTypeDefinitions with + | Some(TUnion(_, cases)) -> name, cases |> Map.tryFind caseName |> Option.flatten + | _ -> qualifier |> Option.defaultValue name, None + | TUnion(name, cases) -> name, cases |> Map.tryFind caseName |> Option.flatten + | _ -> + resolvedTypeDefinitions + |> Map.toSeq + |> Seq.tryPick (fun (name, definition) -> + match definition with + | TUnion(_, cases) when Map.containsKey caseName cases -> Some(name, cases.[caseName]) + | _ -> None) + |> Option.defaultValue (qualifier |> Option.defaultValue "", None) + make (ResolvedUnionPattern(unionName, caseName, Option.map (resolvePattern (payloadType |> Option.defaultValue (Types.freshVar()))) payload)) + | PTypeRef(typeRef, _) -> make (ResolvedTypePattern(annotationTypeFromRef resolvedTypeDefinitions span typeRef)) + let rec resolveExpression (expression: Expr) : ResolvedExpression = let expressionType = finalType expression - let children = - match expression with - | ELambda(_, body, _) | ESome(body, _) | EParen(body, _) | ERaise(body, _) -> [ body ] - | EApply(left, right, _) | EBinOp(_, left, right, _) | EAppend(left, right, _) | ECons(left, right, _) - | EIndexGet(left, right, _) | ERange(left, right, _) -> [ left; right ] - | EIf(condition, whenTrue, whenFalse, _) -> [ condition; whenTrue; whenFalse ] - | ELet(_, binding, body, _, _, _) | ELetPattern(_, binding, body, _) -> [ binding; body ] - | ELetRecGroup(bindings, body, _) -> - (bindings - |> List.map (fun (_, parameters, _, value, span) -> - Seq.foldBack (fun parameter inner -> ELambda(parameter, inner, span)) parameters value)) - @ [ body ] - | EFor(_, source, body, _) -> [ source; body ] - | EMatch(target, cases, _) -> target :: (cases |> List.collect (fun (_, guard, body, _) -> (Option.toList guard) @ [ body ])) - | EList(items, _) | ETuple(items, _) -> items - | ERecord(fields, _) | EStructuralRecord(fields, _) -> List.map snd fields - | EMap(entries, _) -> entries |> List.collect (function MEKeyValue(key, value) -> [ key; value ] | MESpread value -> [ value ]) - | ERecordUpdate(target, fields, _) | EStructuralRecordUpdate(target, fields, _) -> target :: List.map snd fields - | EFieldGet(target, _, _) -> [ target ] - | EInterpolatedString(parts, _) -> parts |> List.choose (function IPText _ -> None | IPExpr item -> Some item) - | EUnit _ | ELiteral _ | EVar _ | ENone _ | ETypeOf _ | ENameOf _ -> [] - |> List.map resolveExpression - let reference = - let span = Ast.spanOfExpr expression - resolvedReferences |> List.tryFind (fun item -> item.Span = span) + let reference = referenceAt expression let primitive = match expression with | EBinOp(op, left, _, _) -> @@ -1678,6 +1882,87 @@ module TypeInfer = | ("&&" | "||"), _ -> Some(BooleanOperation op) | _ -> None | _ -> None + let node : ResolvedExpressionNode = + match expression with + | EUnit _ -> ResolvedConstant ResolvedUnit + | ELiteral(value, _) -> ResolvedConstant(resolvedLiteral value) + | EVar _ -> ResolvedReference(Option.get reference) + | EParen(inner, _) -> (resolveExpression inner).Node + | ELambda(parameter, body, _) -> + let parameterType = match expressionType with TFun(argument, _) -> argument | _ -> Types.freshVar() + ResolvedLambda(bindingAt ParameterSymbol parameter.Name parameter.Span parameterType, resolveExpression body) + | EApply(fn, argument, _) -> ResolvedApply(resolveExpression fn, resolveExpression argument) + | EIf(condition, whenTrue, whenFalse, _) -> ResolvedIf(resolveExpression condition, resolveExpression whenTrue, resolveExpression whenFalse) + | EBinOp("@", left, right, _) | EAppend(left, right, _) -> ResolvedAppend(resolveExpression left, resolveExpression right) + | EBinOp(_, left, right, _) -> ResolvedPrimitive(Option.get primitive, [ resolveExpression left; resolveExpression right ]) + | ECons(head, tail, _) -> ResolvedCons(resolveExpression head, resolveExpression tail) + | ETuple(items, _) -> ResolvedTuple(List.map resolveExpression items) + | EList(items, _) -> ResolvedList(List.map resolveExpression items) + | ERecord(fields, _) | EStructuralRecord(fields, _) -> ResolvedRecord(fields |> List.map (fun (name, value) -> name, resolveExpression value)) + | EFieldGet(_, _, _) when reference.IsSome -> ResolvedReference(Option.get reference) + | EFieldGet(target, name, _) -> ResolvedFieldGet(resolveExpression target, name) + | ESome(item, _) -> ResolvedSome(resolveExpression item) + | ENone _ -> ResolvedNone + | ELet(name, value, body, recursive, _, span) -> + let valueType = finalType value + ResolvedLet(bindingAt LocalSymbol name span valueType, recursive, resolveExpression value, resolveExpression body) + | ELetPattern(pattern, value, body, _) -> + let resolvedValue = resolveExpression value + ResolvedMatch(resolvedValue, [ { Pattern = resolvePattern resolvedValue.Type pattern; Guard = None; Body = resolveExpression body; Span = Ast.spanOfPattern pattern } ]) + | ELetRecGroup(bindings, body, _) -> + let values = + bindings |> List.map (fun (name, parameters, _, value, span) -> + let folded = Seq.foldBack (fun parameter inner -> ELambda(parameter, inner, span)) parameters value + let resolvedValue = resolveExpression folded + { Binding = bindingAt LocalSymbol name span resolvedValue.Type; Value = resolvedValue }) + ResolvedLetRecursiveGroup(values, resolveExpression body) + | ERaise(value, _) -> ResolvedFail(resolveExpression value) + | EFor(name, source, body, span) -> + let resolvedSource = resolveExpression source + let itemType = match resolvedSource.Type with TList item -> item | _ -> Types.freshVar() + ResolvedForEach(bindingAt LocalSymbol name span itemType, resolvedSource, resolveExpression body) + | EMatch(target, cases, _) -> + let resolvedTarget = resolveExpression target + let resolvedCases = + cases |> List.map (fun (pattern, guard, body, caseSpan) -> + { Pattern = resolvePattern resolvedTarget.Type pattern + Guard = Option.map resolveExpression guard + Body = resolveExpression body + Span = caseSpan }) + ResolvedMatch(resolvedTarget, resolvedCases) + | ERange(first, last, _) -> ResolvedRange(resolveExpression first, resolveExpression last) + | ERecordUpdate(target, fields, _) -> + ResolvedRecordUpdate(true, resolveExpression target, fields |> List.map (fun (name, value) -> name, resolveExpression value)) + | EStructuralRecordUpdate(target, fields, _) -> + ResolvedRecordUpdate(false, resolveExpression target, fields |> List.map (fun (name, value) -> name, resolveExpression value)) + | EIndexGet(target, index, _) -> ResolvedIndexGet(resolveExpression target, resolveExpression index) + | ENameOf(name, _) -> ResolvedConstant(ResolvedString name) + | ETypeOf(name, _) -> ResolvedTypeToken(Map.find name resolvedTypeDefinitions) + | EInterpolatedString(parts, _) -> + parts + |> List.map (function IPText text -> ResolvedTextPart text | IPExpr item -> ResolvedValuePart(resolveExpression item)) + |> ResolvedInterpolate + | EMap(entries, _) -> + entries + |> List.map (function MEKeyValue(key, value) -> ResolvedKeyValue(resolveExpression key, resolveExpression value) | MESpread value -> ResolvedSpread(resolveExpression value)) + |> ResolvedMap + let children = + match node with + | ResolvedLambda(_, body) | ResolvedSome body | ResolvedFail body -> [ body ] + | ResolvedApply(left, right) | ResolvedCons(left, right) | ResolvedAppend(left, right) + | ResolvedRange(left, right) | ResolvedIndexGet(left, right) -> [ left; right ] + | ResolvedIf(condition, whenTrue, whenFalse) -> [ condition; whenTrue; whenFalse ] + | ResolvedPrimitive(_, values) | ResolvedTuple values | ResolvedList values -> values + | ResolvedRecord fields -> List.map snd fields + | ResolvedFieldGet(target, _) -> [ target ] + | ResolvedLet(_, _, value, body) -> [ value; body ] + | ResolvedLetRecursiveGroup(bindings, body) -> (bindings |> List.map _.Value) @ [ body ] + | ResolvedMatch(target, cases) -> target :: (cases |> List.collect (fun case -> Option.toList case.Guard @ [ case.Body ])) + | ResolvedRecordUpdate(_, target, fields) -> target :: List.map snd fields + | ResolvedInterpolate parts -> parts |> List.choose (function ResolvedTextPart _ -> None | ResolvedValuePart value -> Some value) + | ResolvedMap entries -> entries |> List.collect (function ResolvedKeyValue(key, value) -> [ key; value ] | ResolvedSpread value -> [ value ]) + | ResolvedForEach(_, source, body) -> [ source; body ] + | ResolvedConstant _ | ResolvedReference _ | ResolvedNone | ResolvedTypeToken _ -> [] let coercion = match expression with | EApply(fn, argument, _) -> @@ -1686,8 +1971,8 @@ module TypeInfer = | TFun((TRecord _ as targetType), _), (TNamed source as sourceType) -> Some { SourceType = sourceType; TargetType = targetType; TargetName = Some source } | _ -> None | _ -> None - { Source = expression; Type = expressionType; Span = Ast.spanOfExpr expression; Reference = reference - Primitive = primitive; RecordCoercion = coercion; Children = children } + { Type = expressionType; Span = Ast.spanOfExpr expression; Reference = reference + Primitive = primitive; RecordCoercion = coercion; Node = node; Children = children } let resolvedStatements = typed |> List.map (fun statement -> let roots = @@ -1713,7 +1998,74 @@ module TypeInfer = |> Option.map (fun import -> [ { Name = "Import"; Arguments = [ import.ModulePath; import.MemberName ]; Span = import.Span } ]) |> Option.defaultValue [] | _ -> [] - { Source = statement; Attributes = attributes; Export = export; Expressions = List.map resolveExpression roots }) + let patterns = + match statement with + | TSLetPattern(pattern, expression, _, _, _) -> [ resolvePattern (finalType expression) pattern ] + | _ -> [] + let resolvedRoots = List.map resolveExpression roots + let declarationBinding name span bindingType = bindingAt DeclarationSymbol name span bindingType + let declarationScheme name span fallback = + (symbolAt DeclarationSymbol name span).Scheme |> Option.defaultValue fallback + let importMetadata (import: BackendImport) : ResolvedBackendImport = + { ModulePath = import.ModulePath; MemberName = import.MemberName; Span = import.Span } + let node = + match statement with + | TSType definition -> + ResolvedTypeDeclaration + { Binding = bindingAt TypeSymbol definition.Name definition.Span (TNamed definition.Name) + IsRecursive = definition.IsRecursive + Definition = resolvedTypeDefinitions.[definition.Name] + Fields = + definition.Fields + |> List.map (fun (name, fieldType) -> name, typeFromRef declarations [ definition.Name ] fieldType) + Cases = + definition.Cases + |> List.map (fun (name, payload) -> name, Option.map (typeFromRef declarations [ definition.Name ]) payload) + Import = Option.map importMetadata definition.Import } + | TSLet(name, _, inferred, recursive, _, span) -> + ResolvedValueDeclaration + { Binding = declarationBinding name span (normalizeTypeVariables inferred) + Scheme = declarationScheme name span (Types.generalize Map.empty inferred) + IsRecursive = recursive + Value = List.exactlyOne resolvedRoots } + | TSLetPattern(_, _, bindings, _, span) -> + let resolvedBindings = + bindings + |> Map.toList + |> List.map (fun (name, bindingType) -> + let normalized = normalizeTypeVariables bindingType + declarationBinding name span normalized, + declarationScheme name span (Types.generalize Map.empty normalized)) + ResolvedPatternDeclaration + { Pattern = List.exactlyOne patterns + Value = List.exactlyOne resolvedRoots + Bindings = resolvedBindings } + | TSLetRecGroup(bindings, _, _) -> + List.zip bindings resolvedRoots + |> List.map (fun ((name, _, inferred, span), value) -> + { Binding = declarationBinding name span (normalizeTypeVariables inferred) + Scheme = declarationScheme name span (Types.generalize Map.empty inferred) + IsRecursive = true + Value = value }) + |> ResolvedRecursiveDeclarations + | TSBackendImport(name, importedType, import, span) -> + let normalized = normalizeTypeVariables importedType + ResolvedImportedDeclaration + { Binding = declarationBinding name span normalized + Scheme = declarationScheme name span (Forall([], normalized)) + Import = importMetadata import } + | TSExpr _ -> ResolvedExpressionStatement(List.exactlyOne resolvedRoots) + { Span = + match statement with + | TSType definition -> definition.Span + | TSLet(_, _, _, _, _, span) | TSLetPattern(_, _, _, _, span) + | TSLetRecGroup(_, _, span) | TSBackendImport(_, _, _, span) -> span + | TSExpr expression -> expression.Span + Attributes = attributes + Export = export + Node = node + Patterns = patterns + Expressions = resolvedRoots }) { TypedProgram = typed ResolvedStatements = resolvedStatements ExpressionTypes = expressions diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index 3f168ae..7de3a54 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -116,6 +116,24 @@ type HirTests() = let identity = compiled.Hir.Declarations |> List.find (fun declaration -> declaration.Name = "identity") Assert.That(identity.Scheme, Is.EqualTo(Forall([ 0 ], TFun(TVar 0, TVar 0)))) + [] + member _.``HIR lowering consumes only the semantic resolved program`` () = + let program = Parser.parseProgram "let add x y = x + y\nadd 1 2" + let inferenceExterns = + CompilerExternCatalog.defaults + |> List.map (fun descriptor -> + ({ Name = descriptor.Name + Scheme = descriptor.Scheme + Arity = descriptor.Arity + Impl = fun _ _ -> VUnit }: ExternalFunction)) + let resolved = TypeInfer.inferResolvedProgramWithExterns inferenceExterns program + let detached = { resolved with TypedProgram = []; ExpressionTypes = [] } + match HirLowering.lower CompilerExternCatalog.defaults detached with + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Ok hir -> + Assert.That(hir.Declarations |> List.exists (fun declaration -> declaration.Name = "add"), Is.True) + Assert.That(Hir.format hir, Does.Contain("primitive IntAdd")) + [] member _.``HIR attaches structured export and import attributes`` () = let source = "[] type Person = { Name: string }\n[] let answer = 42" diff --git a/tests/FScript.Conformance.Tests/ConformanceTests.fs b/tests/FScript.Conformance.Tests/ConformanceTests.fs index fa261f6..1d95d7e 100644 --- a/tests/FScript.Conformance.Tests/ConformanceTests.fs +++ b/tests/FScript.Conformance.Tests/ConformanceTests.fs @@ -112,6 +112,7 @@ type ConformanceTests() = [ value + amount\napplyTwice (add 2) 3")>] [] [ head | _ -> 0")>] + [ true | _ -> false")>] member _.``Legacy LIR artifact and Go results conform`` (source: string) = let expected = Harness.legacy source match Harness.lir source with diff --git a/tests/FScript.Language.Tests/ResolvedInferenceTests.fs b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs index 3eaaba4..c30bb6a 100644 --- a/tests/FScript.Language.Tests/ResolvedInferenceTests.fs +++ b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs @@ -62,6 +62,28 @@ type ResolvedInferenceTests() = Assert.That(expressions |> List.forall (fun expression -> expression.Type <> TVar -1), Is.True) Assert.That(expressions |> List.exists (fun expression -> expression.Primitive = Some TypeInfer.IntegerAddition), Is.True) Assert.That(expressions |> List.exists (fun expression -> expression.Reference |> Option.exists (fun reference -> reference.Name = "add")), Is.True) + Assert.That(expressions |> List.exists (fun expression -> match expression.Node with TypeInfer.ResolvedPrimitive(TypeInfer.IntegerAddition, _) -> true | _ -> false), Is.True) + Assert.That(expressions |> List.exists (fun expression -> match expression.Node with TypeInfer.ResolvedReference reference when reference.Name = "add" && reference.Kind = TypeInfer.DeclarationSymbol -> true | _ -> false), Is.True) + + [] + member _.``Resolved patterns reuse one symbol for repeated bindings`` () = + let source = "match { [\"a\"] = \"a\" } with\n | { [same] = same } -> true\n | _ -> false" + let resolved = Helpers.parse source |> TypeInfer.inferResolvedProgram + let rec bindingIds (pattern: TypeInfer.ResolvedPattern) = + match pattern.Node with + | TypeInfer.ResolvedPatternBinding binding -> [ binding.SymbolId ] + | TypeInfer.ResolvedMapPattern(clauses, tail) -> + (clauses |> List.collect (fun (key, value) -> bindingIds key @ bindingIds value)) + @ (tail |> Option.map bindingIds |> Option.defaultValue []) + | _ -> [] + let ids = + resolved.ResolvedStatements + |> List.collect (fun statement -> + match statement.Node with + | TypeInfer.ResolvedExpressionStatement { Node = TypeInfer.ResolvedMatch(_, cases) } -> cases |> List.collect (_.Pattern >> bindingIds) + | _ -> []) + Assert.That(ids.Length, Is.EqualTo(2)) + Assert.That(ids |> List.distinct |> List.length, Is.EqualTo(1)) [] member _.``Resolved applications retain record coercion evidence`` () = From 195df67a5c1ec7bd30fff5237e44a88cb4cf6559 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 20:47:36 +0200 Subject: [PATCH 051/133] fix(go): preserve intrinsic and result semantics --- CHANGELOG.md | 2 + docs/specs/compiler-semantics-baseline.md | 3 +- src/FScript.Backend.Go/GoBackend.fs | 4 +- src/FScript.Backend.Go/Runtime/runtime.go | 125 +++++++++++++++--- src/FScript.Interpreter.Lir/LirInterpreter.fs | 10 +- .../GoBackendTests.fs | 18 +-- .../ConformanceTests.fs | 25 ++-- 7 files changed, 149 insertions(+), 38 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5421d49..69a18ee 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Generated Go now preserves FScript UTF-16 string indices, strict Boolean parsing, and reference-compatible result formatting across the complete intrinsic conformance catalog. + - HIR lowering now consumes parser-independent semantic resolved statements, expressions, patterns, references, and declaration schemes. - Compilation now preserves every default runtime external as typed host linkage with reachability-based backend diagnostics. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 7ae126f..47d3373 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -45,7 +45,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Record updates evaluate the base once and then updates in source order. Ranges, loops, interpolation, indexing, raise, append, and cons preserve the evaluator's existing evaluation order and failures. - Equality is structural for primitives, tuples, records, maps, options, lists, unions, and type tokens, recursively. Functions, tasks, constructors, and extern handles are not structurally equal. Backend host-language comparability must not leak into this behavior. - Cross-runtime tests compare the legacy evaluator with artifact-round-tripped LIR execution and generated Go for successful values and failure categories. Named records passed through typed function boundaries keep record field projection semantics; their nominal annotation must not select a union runtime representation. -- Cross-runtime aggregate formatting quotes nested strings while leaving top-level strings and ordered-map string keys unquoted; lists use compact semicolon separators and tuples retain comma-space separators. +- Cross-runtime result formatting quotes strings at every nesting level, renders unit as `()`, renders ordered maps as `map { key => value }`, uses compact semicolon separators for lists, and retains comma-space separators for tuples. - HIR and LIR are accepted only when artifact-local references resolve, source spans are valid, declaration and expression types agree, calls and control-flow nodes have consistent signatures, aggregate representations match their semantic types, and referenced primitives/imports are available and complete. - Task signatures and host linkage remain representable in compiled artifacts. An unused task function handle is inert; fully applying `Task.spawn` or `Task.await` in LIR reports `FSI3021`, while a Go program whose reachable declaration graph requires a task reports `FSB4004`. - Compiler extern input is declarative: it carries a name, type scheme, arity, and linkage classification, never an executable `.NET` delegate. Host execution values remain confined to the legacy/runtime API. @@ -59,6 +59,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Scalar conversions, string helpers, `List`, `Option`, and `Map` functions are protected builtin symbols with explicit polymorphic schemes. They are initial compiler/runtime intrinsics even where a future implementation can compile their FScript source. - The LIR interpreter and Go runtime execute the complete pure builtin catalog, including higher-order list, option, and ordered-map functions. Callback invocation remains curried and follows source collection order. +- Generated Go implements FScript string indexing and substring ranges in UTF-16 code units, matching the reference evaluator and Fable string model rather than Go byte offsets. Boolean parsing accepts only case-insensitive `true` and `false`, matching the reference runtime. - `Console.writeLine`, `Console.readLine`, and the CLI `Env` value are candidate portable runtime primitives. Filesystem, regex, hashing, GUID, JSON, and XML functions remain host externs initially. - Compiled `Env` is a runtime-context intrinsic. The LIR interpreter uses its explicit script name and arguments; generated Go uses the executable base name and `os.Args[1:]`. - `Task.spawn` and `Task.await` are host externs with tracked-task semantics, background error propagation, and an error for unawaited tasks. HIR may represent them before LIR interpreters or backends support them. diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index 04b6ed2..18f6124 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -54,7 +54,7 @@ module GoBackend = | TFloat -> "float64" | TBool -> "bool" | TString -> "string" - | TTypeToken -> "string" + | TTypeToken -> "fsruntime.TypeToken" | TVar _ -> "any" | TFun _ -> "fsruntime.Closure" | TTuple _ -> "fsruntime.Tuple" @@ -472,7 +472,7 @@ module GoBackend = let itemCode = cast itemType "rawValue" Ok $"func() struct{{}} {{\n\tfor _, rawValue := range fsruntime.List({sourceCode}) {{\n\t\t{name} := {itemCode}\n\t\t_ = {indent (indent bodyCode)}\n\t}}\n\treturn struct{{}}{{}}\n}}()" | Error error, _ | _, Error error -> Error error - | Lir.TypeToken tokenType -> Ok(System.Text.Json.JsonSerializer.Serialize(Types.typeToString tokenType)) + | Lir.TypeToken tokenType -> Ok($"fsruntime.TypeToken{{Value: {System.Text.Json.JsonSerializer.Serialize(Types.typeToString tokenType)}}}") | Lir.Operation _ -> Error "Invalid operation arity" and private collect<'a> (results: Result<'a, string> list) : Result<'a list, string> = diff --git a/src/FScript.Backend.Go/Runtime/runtime.go b/src/FScript.Backend.Go/Runtime/runtime.go index 612d577..34a512e 100644 --- a/src/FScript.Backend.Go/Runtime/runtime.go +++ b/src/FScript.Backend.Go/Runtime/runtime.go @@ -6,6 +6,7 @@ import ( "sort" "strconv" "strings" + "unicode/utf16" ) type Option struct { @@ -13,6 +14,7 @@ type Option struct { Value any } type Tuple []any +type TypeToken struct{ Value string } type Union struct { TypeName string CaseName string @@ -34,6 +36,35 @@ type MapPart struct { Map OrderedMap } +func utf16Index(value string, needle string) int64 { + valueUnits := utf16.Encode([]rune(value)) + needleUnits := utf16.Encode([]rune(needle)) + if len(needleUnits) == 0 { + return 0 + } + for start := 0; start+len(needleUnits) <= len(valueUnits); start++ { + matches := true + for offset, unit := range needleUnits { + if valueUnits[start+offset] != unit { + matches = false + break + } + } + if matches { + return int64(start) + } + } + return -1 +} + +func utf16Substring(value string, start int64, length int64) Option { + units := utf16.Encode([]rune(value)) + if start < 0 || length < 0 || start+length > int64(len(units)) { + return None() + } + return Some(string(utf16.Decode(units[start : start+length]))) +} + func Unit(value any) struct{} { return struct{}{} } func Int(value any) int64 { return value.(int64) } func Float(value any) float64 { return value.(float64) } @@ -264,31 +295,31 @@ func Intrinsic(name string, args []any) any { } return Some(value) case "Bool.tryParse": - value, error := strconv.ParseBool(String(args[0])) - if error != nil { + switch strings.ToLower(String(args[0])) { + case "true": + return Some(true) + case "false": + return Some(false) + default: return None() } - return Some(value) case "Int.toString", "Float.toString", "Bool.toString": return Format(args[0]) case "String.replace": return strings.ReplaceAll(String(args[2]), String(args[0]), String(args[1])) case "String.indexOf": - index := strings.Index(String(args[1]), String(args[0])) + index := utf16Index(String(args[1]), String(args[0])) if index < 0 { return None() } - return Some(int64(index)) + return Some(index) case "String.toLower": return strings.ToLower(String(args[0])) case "String.toUpper": return strings.ToUpper(String(args[0])) case "String.substring": start, length, value := Int(args[0]), Int(args[1]), String(args[2]) - if start < 0 || length < 0 || start+length > int64(len(value)) { - return None() - } - return Some(value[start : start+length]) + return utf16Substring(value, start, length) case "String.concat": values := List(args[1]) parts := make([]string, len(values)) @@ -366,10 +397,11 @@ func formatValue(value any, nested bool) string { } return "false" case string: - if nested { - return strconv.Quote(item) - } - return item + return strconv.Quote(item) + case TypeToken: + return "" + case struct{}: + return "()" case Option: if !item.HasValue { return "None" @@ -378,9 +410,9 @@ func formatValue(value any, nested bool) string { case OrderedMap: parts := make([]string, len(item.Entries)) for index, entry := range item.Entries { - parts[index] = "[" + formatValue(entry.Key, false) + "] = " + formatValue(entry.Value, true) + parts[index] = formatValue(entry.Key, true) + " => " + formatValue(entry.Value, true) } - return "{ " + strings.Join(parts, "; ") + " }" + return "map { " + strings.Join(parts, "; ") + " }" case []any: parts := make([]string, len(item)) for index, value := range item { @@ -416,8 +448,69 @@ func formatValue(value any, nested bool) string { func Interpolate(values []any) string { var result strings.Builder for _, value := range values { - result.WriteString(Format(value)) + result.WriteString(formatInterpolationValue(value)) } return result.String() } + +func formatInterpolationValue(value any) string { + if item, ok := TryUnion(value); ok { + if !item.HasValue { + return item.CaseName + } + return item.CaseName + " " + formatInterpolationValue(item.Value) + } + switch item := value.(type) { + case string: + return item + case TypeToken: + return "" + case struct{}: + return "()" + case Option: + if !item.HasValue { + return "None" + } + return "Some " + formatInterpolationValue(item.Value) + case OrderedMap: + parts := make([]string, len(item.Entries)) + for index, entry := range item.Entries { + key, ok := entry.Key.(string) + if ok { + parts[index] = strconv.Quote(key) + " => " + formatInterpolationValue(entry.Value) + } else { + parts[index] = formatInterpolationValue(entry.Key) + " => " + formatInterpolationValue(entry.Value) + } + } + return "map { " + strings.Join(parts, "; ") + " }" + case []any: + parts := make([]string, len(item)) + for index, value := range item { + parts[index] = formatInterpolationValue(value) + } + return "[" + strings.Join(parts, ";") + "]" + case Tuple: + parts := make([]string, len(item)) + for index, value := range item { + parts[index] = formatInterpolationValue(value) + } + return "(" + strings.Join(parts, ", ") + ")" + case map[string]any: + names := make([]string, 0, len(item)) + for name := range item { + names = append(names, name) + } + sort.Strings(names) + parts := make([]string, len(names)) + for index, name := range names { + parts[index] = name + " = " + formatInterpolationValue(item[name]) + } + return "{ " + strings.Join(parts, "; ") + " }" + default: + if record, ok := TryRecord(value); ok { + return formatInterpolationValue(record) + } + return fmt.Sprint(value) + } +} func Fail(value any) { panic(fmt.Sprint(value)) } diff --git a/src/FScript.Interpreter.Lir/LirInterpreter.fs b/src/FScript.Interpreter.Lir/LirInterpreter.fs index 423019a..2544cc5 100644 --- a/src/FScript.Interpreter.Lir/LirInterpreter.fs +++ b/src/FScript.Interpreter.Lir/LirInterpreter.fs @@ -64,7 +64,15 @@ module LirInterpreter = | VOption None -> "None" | VOption(Some value) -> $"Some {formatValue value}" | VUnion(_, caseName, None) -> caseName | VUnion(_, caseName, Some value) -> $"{caseName} {formatValue value}" | VMap entries -> - entries |> List.map (fun (key, value) -> $"[{formatValue key}] = {formatValue value}") |> String.concat "; " |> fun value -> "{ " + value + " }" + entries + |> List.map (fun (key, value) -> + let renderedKey = + match key with + | VString text -> "\"" + text.Replace("\\", "\\\\").Replace("\"", "\\\"") + "\"" + | _ -> formatValue key + $"{renderedKey} => {formatValue value}") + |> String.concat "; " + |> fun value -> "map { " + value + " }" | VClosure _ -> "" | VIntrinsic _ -> "" | VExternal _ -> "" diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index cb01968..9911864 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -125,16 +125,16 @@ type GoBackendTests() = [ x\n | _ -> 42", "42")>] [] [] - [] + [] [ 0\n | Circle r -> r\nradius (Circle 3)", "3")>] [] [] - [] + [] [] [] [] [] - [] + [")>] [ value + 1) [1; 2]", "[2;3]")>] [ if value > 1 then Some value else None) [1; 2; 3]", "[2;3]")>] [ [value; value]) [1; 2]", "[1;1;2;2]")>] @@ -149,19 +149,19 @@ type GoBackendTests() = [ value > 1) [1; 2; 3]", "[2;3]")>] [ value > 1) [1; 2; 3]", "Some 2")>] [] - [] - [ value + 1) (Map.filter (fun key value -> value > 1) (Map.ofList [(\"b\", 2); (\"a\", 1)]))", "{ [b] = 3 }")>] - [ if value > 1 then Some (value + 1) else None) { [\"a\"] = 1; [\"b\"] = 2 }", "{ [b] = 3 }")>] - [] + [ 1; \"b\" => 2 }")>] + [ value + 1) (Map.filter (fun key value -> value > 1) (Map.ofList [(\"b\", 2); (\"a\", 1)]))", "map { \"b\" => 3 }")>] + [ if value > 1 then Some (value + 1) else None) { [\"a\"] = 1; [\"b\"] = 2 }", "map { \"b\" => 3 }")>] + [ 2 }")>] [] [] [ value\n | {} -> 0", "1")>] [ true\n | _ -> false", "true")>] [] [] - [] let upper : string -> string\nupper \"hello\"", "HELLO")>] + [] let upper : string -> string\nupper \"hello\"", "\"HELLO\"")>] [] - [] + [] [] [] [] diff --git a/tests/FScript.Conformance.Tests/ConformanceTests.fs b/tests/FScript.Conformance.Tests/ConformanceTests.fs index 1d95d7e..982aa47 100644 --- a/tests/FScript.Conformance.Tests/ConformanceTests.fs +++ b/tests/FScript.Conformance.Tests/ConformanceTests.fs @@ -10,24 +10,27 @@ open NUnit.Framework module private Harness = type Outcome = { Value: string; Stdout: string list } - let rec private formatLirValue nested = function + let private quoteString (value: string) = + "\"" + value.Replace("\\", "\\\\").Replace("\"", "\\\"").Replace("\r", "\\r").Replace("\n", "\\n").Replace("\t", "\\t") + "\"" + + let rec private formatLirValue = function | LirValue.VUnit -> "()" | LirValue.VInt value -> string value | LirValue.VFloat value -> value.ToString("R", Globalization.CultureInfo.InvariantCulture) | LirValue.VBool value -> string value |> _.ToLowerInvariant() - | LirValue.VString value -> if nested then System.Text.Json.JsonSerializer.Serialize(value) else value + | LirValue.VString value -> quoteString value | LirValue.VOption None -> "None" - | LirValue.VOption(Some value) -> "Some " + formatLirValue true value - | LirValue.VList values -> "[" + (values |> List.map (formatLirValue true) |> String.concat ";") + "]" - | LirValue.VTuple values -> "(" + (values |> List.map (formatLirValue true) |> String.concat ", ") + ")" - | LirValue.VRecord fields -> "{ " + (fields |> Map.toList |> List.map (fun (name, value) -> name + " = " + formatLirValue true value) |> String.concat "; ") + " }" + | LirValue.VOption(Some value) -> "Some " + formatLirValue value + | LirValue.VList values -> "[" + (values |> List.map formatLirValue |> String.concat ";") + "]" + | LirValue.VTuple values -> "(" + (values |> List.map formatLirValue |> String.concat ", ") + ")" + | LirValue.VRecord fields -> "{ " + (fields |> Map.toList |> List.map (fun (name, value) -> name + " = " + formatLirValue value) |> String.concat "; ") + " }" | LirValue.VUnion(_, caseName, None) -> caseName - | LirValue.VUnion(_, caseName, Some value) -> caseName + " " + formatLirValue true value - | LirValue.VMap entries -> "{ " + (entries |> List.map (fun (key, value) -> "[" + formatLirValue false key + "] = " + formatLirValue true value) |> String.concat "; ") + " }" + | LirValue.VUnion(_, caseName, Some value) -> caseName + " " + formatLirValue value + | LirValue.VMap entries -> "map { " + (entries |> List.map (fun (key, value) -> formatLirValue key + " => " + formatLirValue value) |> String.concat "; ") + " }" | LirValue.VTypeToken value -> "" | LirValue.VClosure _ | LirValue.VIntrinsic _ | LirValue.VExternal _ | LirValue.VTargetImport _ -> "" | LirValue.VRuntimeContext -> "" - let formatLir value = formatLirValue false value + let formatLir value = formatLirValue value let compile source = match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with @@ -113,6 +116,10 @@ type ConformanceTests() = [] [ head | _ -> 0")>] [ true | _ -> false")>] + [] + [ value + 1) [1; 2], List.iter (fun value -> ignore value) [1; 2], List.choose (fun value -> if value > 1 then Some value else None) [1; 2], List.collect (fun value -> [value; value]) [1; 2], List.exists (fun value -> value = 2) [1; 2], List.contains 2 [1; 2], List.rev [1; 2], List.distinct [1; 1; 2], List.fold (fun state value -> state + value) 0 [1; 2], List.filter (fun value -> value > 1) [1; 2], List.length [1; 2], List.tryFind (fun value -> value > 1) [1; 2], List.tryGet (fun value -> value = 2) [1; 2], List.tryItem 1 [1; 2], List.tryHead [1; 2], List.tail [1; 2], List.append [1] [2])")>] + [ 8) None, Option.isNone None, Option.isSome (Some 1), Option.map (fun value -> value + 1) (Some 1))")>] + [ state + value) 0 (Map.ofList [(\"a\", 1); (\"b\", 2)]), Map.filter (fun key value -> value > 1) (Map.ofList [(\"a\", 1); (\"b\", 2)]), Map.choose (fun key value -> if value > 1 then Some value else None) (Map.ofList [(\"a\", 1); (\"b\", 2)]), Map.map (fun value -> value + 1) (Map.ofList [(\"a\", 1)]), Map.iter (fun key value -> ignore value) (Map.ofList [(\"a\", 1)]), Map.remove \"a\" (Map.ofList [(\"a\", 1); (\"b\", 2)]))")>] member _.``Legacy LIR artifact and Go results conform`` (source: string) = let expected = Harness.legacy source match Harness.lir source with From 7937f3922a4dda253447c54169a45261729a4a0d Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 20:55:07 +0200 Subject: [PATCH 052/133] feat(language): retain custom declaration attributes --- CHANGELOG.md | 2 + docs/architecture/typed-ir-and-backends.md | 2 +- docs/specs/compiler-semantics-baseline.md | 1 + docs/specs/syntax-and-indentation.md | 3 +- .../LanguageServer/LspSymbols.fs | 10 +- src/FScript.Language/Ast.fs | 8 ++ src/FScript.Language/IncludeResolver.fs | 14 ++- src/FScript.Language/Parser.fs | 97 ++++++++++--------- src/FScript.Language/TypeInfer.fs | 24 ++++- tests/FScript.Compiler.Tests/HirTests.fs | 9 ++ .../ResolvedInferenceTests.fs | 9 ++ 11 files changed, 120 insertions(+), 59 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 69a18ee..c9fcaf2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Preserved arbitrary structured top-level declaration attributes through resolved inference and HIR. + - Generated Go now preserves FScript UTF-16 string indices, strict Boolean parsing, and reference-compatible result formatting across the complete intrinsic conformance catalog. - HIR lowering now consumes parser-independent semantic resolved statements, expressions, patterns, references, and declaration schemes. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index b6603a4..42d94e0 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -32,7 +32,7 @@ HIR and LIR expose canonical debug printers with stable artifact-local IDs and n HIR declaration schemes are taken directly from the resolved frontend after deterministic type-variable normalization; lowering does not generalize source declarations again. Synthetic declarations may derive a local scheme because they have no frontend declaration counterpart. -Resolved statements expose normalized structured attributes as name, argument list, and source span. HIR attaches these attributes to ordinary value and type declarations while retaining dedicated export and target-import linkage for consumers that need semantic access without reparsing attribute arguments. +Resolved statements expose normalized structured attributes as name, argument list, and source span. The parser retains unknown attributes instead of assigning semantics to them, and HIR attaches them to ordinary value and type declarations while retaining dedicated export and target-import linkage for consumers that need semantic access without reparsing attribute arguments. HIR validation rejects duplicate and unresolved symbols, invalid declaration spans, and schemes inconsistent with their values. LIR validation rechecks artifact-local references and spans, then verifies closure calls, branches and decision guards, primitive signatures, aggregate representations, declaration value types, intrinsic catalog membership, and complete target-import linkage. Artifact deserialization runs the same LIR validator, so malformed JSON cannot bypass compiler invariants. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 47d3373..c9a0d18 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -65,6 +65,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - `Task.spawn` and `Task.await` are host externs with tracked-task semantics, background error propagation, and an error for unawaited tasks. HIR may represent them before LIR interpreters or backends support them. - `[]` is declaration metadata used by embedding hosts and does not change inference. Backend `Import` metadata is separate and will not weaken the declared FScript type. - Both `[]` and `[]` normalize to a structured `Export` attribute in resolved statements and HIR. `Import` attributes retain their two arguments and attribute span on ordinary HIR function/type declarations in addition to dedicated backend linkage. +- Other top-level declaration attributes remain case-sensitive and flow unchanged through resolved statements and HIR; they carry string or identifier arguments but have no evaluator semantics unless a future consumer recognizes them. The compiler-owned runtime catalog is declarative and contains no executable host delegates: diff --git a/docs/specs/syntax-and-indentation.md b/docs/specs/syntax-and-indentation.md index 700514f..862aa2f 100644 --- a/docs/specs/syntax-and-indentation.md +++ b/docs/specs/syntax-and-indentation.md @@ -34,7 +34,7 @@ Backend `Import` requires exactly a target module path and member name, a comple - tuple destructuring is supported for non-`rec` `let` bindings only - `[] let` requires a single identifier binding (tuple patterns are rejected) - `[]` is only valid on top-level `let` bindings - - attribute names are case-sensitive (`[]` is valid, `[]` is not) + - ordinary attribute names are case-sensitive; `export` and `Export` are accepted aliases for the built-in export marker - Conditionals: - `if cond then a else b` - `elif` desugars to nested `if` @@ -252,3 +252,4 @@ let ops = - An imported record schema uses `[] type Name = { Field: type; ... }`. - `Import` takes exactly two non-empty arguments and is valid only on a top-level type declaration or bodyless typed `let`. - Imported record field spelling is preserved exactly for backend adapters; the initial Go backend requires exported, uppercase primitive fields. +- Other top-level type and `let` attributes are retained as structured name, string/identifier arguments, and source span for compiler consumers. Unknown attributes do not change language semantics by themselves. diff --git a/src/FScript.CSharpInterop/LanguageServer/LspSymbols.fs b/src/FScript.CSharpInterop/LanguageServer/LspSymbols.fs index 4eb2cd8..ab20a19 100644 --- a/src/FScript.CSharpInterop/LanguageServer/LspSymbols.fs +++ b/src/FScript.CSharpInterop/LanguageServer/LspSymbols.fs @@ -880,15 +880,19 @@ let queryValues : string -> string -> string list option (entry.Name, entry.Span.Start.Line, entry.Span.Start.Column, entry.Span.End.Line, entry.Span.End.Column, file) for stmt in program do + let rec withoutAttributes = function + | SAttributed(_, inner, _) -> withoutAttributes inner + | statement -> statement let targetNames = - match stmt with + match withoutAttributes stmt with | SLet(name, _, _, _, _, _, _) -> [ name ] | SLetPattern(pattern, _, _, _) -> collectPatternNames pattern | SLetRecGroup(bindings, _, _) -> bindings |> List.map (fun (name, _, _, _, _) -> name) | SBackendImport(name, _, _, _) -> [ name ] | SType _ | SExpr _ - | SImport _ -> [] + | SImport _ + | SAttributed _ -> [] if not targetNames.IsEmpty then match tryInfer stmt with @@ -1197,6 +1201,8 @@ let queryValues : string -> string -> string list option resolveStmts scopesWithNames rest (acc @ inBindings) | SBackendImport(name, _, _, span) -> resolveStmts (pushBindings scopes [ name, span ]) rest acc + | SAttributed(_, inner, _) -> + resolveStmts scopes (inner :: rest) acc | SExpr expr -> let inExpr = resolveExpr scopes expr resolveStmts scopes rest (acc @ inExpr) diff --git a/src/FScript.Language/Ast.fs b/src/FScript.Language/Ast.fs index 0764290..a910425 100644 --- a/src/FScript.Language/Ast.fs +++ b/src/FScript.Language/Ast.fs @@ -99,6 +99,7 @@ and Stmt = | SLetPattern of Pattern * Expr * bool * Span | SLetRecGroup of (string * Param list * TypeRef option * Expr * Span) list * bool * Span | SBackendImport of string * TypeRef * BackendImport * Span + | SAttributed of Attribute list * Stmt * Span | SExpr of Expr type Program = Stmt list @@ -152,3 +153,10 @@ module Ast = | ETypeOf (_, s) -> s | ENameOf (_, s) -> s | EInterpolatedString (_, s) -> s + + let spanOfStmt statement = + match statement with + | SType definition -> definition.Span + | SImport(_, _, span) | SLet(_, _, _, _, _, _, span) | SLetPattern(_, _, _, span) + | SLetRecGroup(_, _, span) | SBackendImport(_, _, _, span) | SAttributed(_, _, span) -> span + | SExpr expression -> spanOfExpr expression diff --git a/src/FScript.Language/IncludeResolver.fs b/src/FScript.Language/IncludeResolver.fs index 393b71c..17209e0 100644 --- a/src/FScript.Language/IncludeResolver.fs +++ b/src/FScript.Language/IncludeResolver.fs @@ -379,9 +379,12 @@ module IncludeResolver = loop [] expr let private rewriteModuleScopedStatements (selfPrefix: string option) (aliases: Map) (statements: Stmt list) : Stmt list = + let rec unwrap = function + | SAttributed(_, statement, _) -> unwrap statement + | statement -> statement let topLevelValueNames = statements - |> List.collect (function + |> List.collect (unwrap >> function | SLet(name, _, _, _, _, _, _) -> [ name ] | SLetPattern(pattern, _, _, _) -> collectPatternBindings pattern |> Set.toList | SLetRecGroup(bindings, _, _) -> bindings |> List.map (fun (name, _, _, _, _) -> name) @@ -390,7 +393,7 @@ module IncludeResolver = let topLevelTypeNames = statements - |> List.choose (function | SType def -> Some def.Name | _ -> None) + |> List.choose (unwrap >> function | SType def -> Some def.Name | _ -> None) |> Set.ofList let rewriteValueName boundNames name = @@ -524,8 +527,8 @@ module IncludeResolver = def.Cases |> List.map (fun (name, payload) -> name, payload |> Option.map (rewriteTypeRef aliases selfPrefix topLevelTypeNames)) } - statements - |> List.map (function + let rec rewriteStatement = function + | SAttributed(attributes, statement, span) -> SAttributed(attributes, rewriteStatement statement, span) | SType def -> SType (rewriteTypeDef def) | SLet(name, args, returnAnnotation, valueExpr, isRec, isExported, span) -> @@ -555,7 +558,8 @@ module IncludeResolver = SBackendImport(qualifySelfName name, rewriteTypeRef aliases selfPrefix topLevelTypeNames signature, import, span) | SExpr expr -> SExpr (rewriteExpr Set.empty expr) - | SImport _ -> failwith "Unexpected unresolved import directive") + | SImport _ -> failwith "Unexpected unresolved import directive" + statements |> List.map rewriteStatement let private expandProgram (rootDirectoryWithSeparator: string) diff --git a/src/FScript.Language/Parser.fs b/src/FScript.Language/Parser.fs index 66ec981..5d9fa13 100644 --- a/src/FScript.Language/Parser.fs +++ b/src/FScript.Language/Parser.fs @@ -1477,6 +1477,8 @@ module Parser = raise (ParseException { Message = "Type declarations are only supported at top level"; Span = def.Span }) | SImport (_, _, span) :: _ -> raise (ParseException { Message = "'import' is only supported at top level"; Span = span }) + | SAttributed(_, _, span) :: _ -> + raise (ParseException { Message = "Attributes are only supported at top level"; Span = span }) desugar (statements |> Seq.toList) and parseLetStmt (isExported: bool) : Stmt = @@ -1611,6 +1613,7 @@ module Parser = let mutable isExported = false let mutable hasAttributes = false let mutable backendImport: Attribute option = None + let structuredAttributes = ResizeArray() let mutable parsingAttributes = true while parsingAttributes do if stream.Peek().Kind = LBracket && stream.PeekAt(1).Kind = Less then @@ -1627,58 +1630,60 @@ module Parser = if backendImport.IsSome then raise (ParseException { Message = "Duplicate attribute 'Import'"; Span = attribute.Span }) backendImport <- Some attribute - | _ -> - raise (ParseException { Message = $"Unknown attribute '{attribute.Name}'"; Span = attribute.Span }) + | _ -> structuredAttributes.Add(attribute) stream.SkipNewlines() else parsingAttributes <- false - match stream.Peek().Kind with - | Hash -> - if hasAttributes || isExported then - raise (ParseException { Message = "Attributes are not supported on 'import' directives"; Span = stream.Peek().Span }) - let hashTok = stream.Next() - match stream.Next().Kind with - | Include -> - raise (ParseException { Message = "'#include' was removed in 0.32.0; use 'import \"file.fss\" as Alias'"; Span = hashTok.Span }) + let statement = + match stream.Peek().Kind with + | Hash -> + if hasAttributes || isExported then + raise (ParseException { Message = "Attributes are not supported on 'import' directives"; Span = stream.Peek().Span }) + let hashTok = stream.Next() + match stream.Next().Kind with + | Include -> + raise (ParseException { Message = "'#include' was removed in 0.32.0; use 'import \"file.fss\" as Alias'"; Span = hashTok.Span }) + | Import -> + raise (ParseException { Message = "'#import' was removed in 0.32.0; use 'import \"file.fss\" as Alias'"; Span = hashTok.Span }) + | _ -> + raise (ParseException { Message = "Unexpected '#'; use 'import \"file.fss\" as Alias'"; Span = hashTok.Span }) | Import -> - raise (ParseException { Message = "'#import' was removed in 0.32.0; use 'import \"file.fss\" as Alias'"; Span = hashTok.Span }) - | _ -> - raise (ParseException { Message = "Unexpected '#'; use 'import \"file.fss\" as Alias'"; Span = hashTok.Span }) - | Import -> - if hasAttributes || isExported then - raise (ParseException { Message = "Attributes are not supported on 'import' directives"; Span = stream.Peek().Span }) - let importTok = stream.Next() - let pathTok = stream.Next() - match pathTok.Kind with - | StringLit path when not (System.String.IsNullOrWhiteSpace path) -> - stream.Expect(As, "Expected 'as' in import directive. Use: import \"file.fss\" as Alias") |> ignore - let aliasTok = stream.ExpectIdent("Expected module alias after 'as'") - let alias = - match aliasTok.Kind with - | Ident value -> value - | _ -> "" - SImport(alias, path, mkSpanFrom importTok.Span aliasTok.Span) - | StringLit _ -> - raise (ParseException { Message = "Import path cannot be empty"; Span = pathTok.Span }) + if hasAttributes || isExported then + raise (ParseException { Message = "Attributes are not supported on 'import' directives"; Span = stream.Peek().Span }) + let importTok = stream.Next() + let pathTok = stream.Next() + match pathTok.Kind with + | StringLit path when not (System.String.IsNullOrWhiteSpace path) -> + stream.Expect(As, "Expected 'as' in import directive. Use: import \"file.fss\" as Alias") |> ignore + let aliasTok = stream.ExpectIdent("Expected module alias after 'as'") + let alias = + match aliasTok.Kind with + | Ident value -> value + | _ -> "" + SImport(alias, path, mkSpanFrom importTok.Span aliasTok.Span) + | StringLit _ -> + raise (ParseException { Message = "Import path cannot be empty"; Span = pathTok.Span }) + | _ -> + raise (ParseException { Message = "Expected string literal import path after 'import'"; Span = pathTok.Span }) + | Module -> + raise (ParseException { Message = "'module' declarations are not supported in scripts"; Span = stream.Peek().Span }) + | Type -> + if isExported then + raise (ParseException { Message = "'[]' is only valid for top-level let bindings"; Span = stream.Peek().Span }) + parseTypeDecl backendImport + | Let -> + match backendImport with + | Some attribute when isExported -> raise (ParseException { Message = "Import and Export cannot be combined initially"; Span = attribute.Span }) + | Some attribute -> parseBackendImport attribute + | None -> parseLetStmt isExported + | _ when hasAttributes -> + raise (ParseException { Message = "Attributes must be followed by a top-level declaration"; Span = stream.Peek().Span }) | _ -> - raise (ParseException { Message = "Expected string literal import path after 'import'"; Span = pathTok.Span }) - | Module -> - raise (ParseException { Message = "'module' declarations are not supported in scripts"; Span = stream.Peek().Span }) - | Type -> - if isExported then - raise (ParseException { Message = "'[]' is only valid for top-level let bindings"; Span = stream.Peek().Span }) - parseTypeDecl backendImport - | Let -> - match backendImport with - | Some attribute when isExported -> raise (ParseException { Message = "Import and Export cannot be combined initially"; Span = attribute.Span }) - | Some attribute -> parseBackendImport attribute - | None -> parseLetStmt isExported - | _ when hasAttributes -> - raise (ParseException { Message = "'[]' must be followed by a top-level let binding"; Span = stream.Peek().Span }) - | _ -> - let expr = parseExpr() - SExpr expr + let expr = parseExpr() + SExpr expr + if structuredAttributes.Count = 0 then statement + else SAttributed(structuredAttributes |> Seq.toList, statement, Ast.spanOfStmt statement) let program = ResizeArray() stream.SkipNewlines() diff --git a/src/FScript.Language/TypeInfer.fs b/src/FScript.Language/TypeInfer.fs index 2816495..9f74b4c 100644 --- a/src/FScript.Language/TypeInfer.fs +++ b/src/FScript.Language/TypeInfer.fs @@ -1352,15 +1352,26 @@ module TypeInfer = | None -> acc loop [] pat + let rec private unwrapStatementAttributes statement = + match statement with + | SAttributed(attributes, inner, _) -> + let nested, unwrapped = unwrapStatementAttributes inner + attributes @ nested, unwrapped + | value -> [], value + + let private unwrapProgramAttributes (program: Program) = + program |> List.map unwrapStatementAttributes |> List.unzip + let private topLevelBindingNames (program: Program) : string list = program - |> List.collect (function + |> List.collect (unwrapStatementAttributes >> snd >> function | SLet(name, _, _, _, _, _, _) -> [ name ] | SLetPattern(pat, _, _, _) -> patternBindingNames pat | SLetRecGroup(bindings, _, _) -> bindings |> List.map (fun (name, _, _, _, _) -> name) | _ -> []) let private inferProgramWithExternsRawAndLocals (captureTelemetry: bool) (externs: ExternalFunction list) (program: Program) : TypedProgram * LocalVariableTypeInfo list * ResolvedExpressionType list = + let _, program = unwrapProgramAttributes program let decls = (BuiltinTypes.typeDefs |> List.map (fun d -> d.Name, d)) @ (program |> List.choose (function | SType d -> Some (d.Name, d) | _ -> None)) @@ -1424,6 +1435,8 @@ module TypeInfer = typed.Add(TSType def) | SImport (_, _, span) -> raise (TypeException { Message = "'import' must be resolved before type inference"; Span = span }) + | SAttributed(_, _, span) -> + raise (TypeException { Message = "Attributed statements must be normalized before type inference"; Span = span }) | SBackendImport(name, signature, import, span) -> let importedType = annotationTypeFromRef typeDefs span signature match importedType with @@ -1622,7 +1635,8 @@ module TypeInfer = typed, locals let inferResolvedProgramWithExterns (externs: ExternalFunction list) (program: Program) : ResolvedProgram = - let typed, _, expressions = inferProgramWithExternsRawAndLocals true externs program + let statementAttributes, semanticProgram = unwrapProgramAttributes program + let typed, _, expressions = inferProgramWithExternsRawAndLocals true externs semanticProgram let expressionTypes = expressions |> List.fold (fun state item -> Map.add item.Expr item.Type state) Map.empty let mutable nextSymbol = 0 let symbols = ResizeArray() @@ -1974,7 +1988,7 @@ module TypeInfer = { Type = expressionType; Span = Ast.spanOfExpr expression; Reference = reference Primitive = primitive; RecordCoercion = coercion; Node = node; Children = children } let resolvedStatements = - typed |> List.map (fun statement -> + (statementAttributes, typed) ||> List.map2 (fun declaredAttributes statement -> let roots = match statement with | TSLet(_, expression, _, _, _, _) | TSLetPattern(_, expression, _, _, _) -> [ expression ] @@ -2061,7 +2075,9 @@ module TypeInfer = | TSLet(_, _, _, _, _, span) | TSLetPattern(_, _, _, _, span) | TSLetRecGroup(_, _, span) | TSBackendImport(_, _, _, span) -> span | TSExpr expression -> expression.Span - Attributes = attributes + Attributes = + declaredAttributes @ attributes + |> List.sortBy (fun attribute -> attribute.Span.Start.Line, attribute.Span.Start.Column) Export = export Node = node Patterns = patterns diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index 7de3a54..8f33487 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -142,6 +142,15 @@ type HirTests() = let answer = compiled.Hir.Declarations |> List.find (fun declaration -> declaration.Name = "answer") Assert.That((answer.Attributes |> List.map _.Name) = [ "Export" ], Is.True) + [] + member _.``HIR retains custom attributes on ordinary declarations`` () = + let source = "[]\ntype Person = { Name: string }\n[] let answer = 42" + let compiled = match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with Ok value -> value | Error diagnostics -> failwith (System.String.Join("; ", diagnostics |> List.map _.Message)) + Assert.That(compiled.Hir.Types.Head.Attributes.Head.Name, Is.EqualTo("BackendHint")) + Assert.That(compiled.Hir.Types.Head.Attributes.Head.Arguments = [ "fast"; "stable" ], Is.True) + let answer = compiled.Hir.Declarations |> List.find (fun declaration -> declaration.Name = "answer") + Assert.That(answer.Attributes.Head.Name, Is.EqualTo("Trace")) + [] member _.``HIR validation rejects a declaration scheme mismatch`` () = let value: Hir.Expression = { Type = TInt; Span = span; Node = Hir.Constant(Hir.Int 1L) } diff --git a/tests/FScript.Language.Tests/ResolvedInferenceTests.fs b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs index c30bb6a..0e6ce4e 100644 --- a/tests/FScript.Language.Tests/ResolvedInferenceTests.fs +++ b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs @@ -115,3 +115,12 @@ type ResolvedInferenceTests() = let imports = resolved.ResolvedStatements |> List.collect _.Attributes |> List.filter (fun attribute -> attribute.Name = "Import") Assert.That(imports.Length, Is.EqualTo(2)) Assert.That(imports |> List.map _.Arguments, Does.Contain([ "strings"; "ToUpper" ])) + + [] + member _.``Resolved statements retain custom structured attributes`` () = + let source = "[]\nlet answer = 42" + let resolved = Helpers.parse source |> TypeInfer.inferResolvedProgram + let attribute = resolved.ResolvedStatements.Head.Attributes |> List.exactlyOne + Assert.That(attribute.Name, Is.EqualTo("BackendHint")) + Assert.That(attribute.Arguments = [ "fast"; "stable" ], Is.True) + Assert.That(attribute.Span.Start.Line, Is.EqualTo(1)) From 0ccb07923c4019c5133df4fdcfa3d2fc6068813e Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 21:00:02 +0200 Subject: [PATCH 053/133] refactor(go): execute serialized pattern plans --- CHANGELOG.md | 2 + docs/architecture/typed-ir-and-backends.md | 4 +- docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Backend.Go/GoBackend.fs | 61 ++++--------------- src/FScript.Backend.Go/Runtime/patterns.go | 11 +--- .../GoBackendTests.fs | 12 ++++ 6 files changed, 32 insertions(+), 59 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c9fcaf2..7fcdcd6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Routed every generated Go pattern plan through one deterministic runtime matcher. + - Preserved arbitrary structured top-level declaration attributes through resolved inference and HIR. - Generated Go now preserves FScript UTF-16 string indices, strict Boolean parsing, and reference-compatible result formatting across the complete intrinsic conformance catalog. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 42d94e0..dacff42 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -22,7 +22,7 @@ Validated LIR consumers are separate leaf assemblies. `FScript.Interpreter.Lir` HIR is language-oriented and contains resolved artifact-local symbol identities, ordinary and imported type declarations, complete semantic types and schemes, source locations, structured export metadata, and explicit nodes for named/structural record conversions. HIR lowering consumes only the semantic resolved statements, expressions, and patterns; it does not traverse `Expr`, `Pattern`, `TypedStmt`, or compatibility expression telemetry. HIR directly reuses the frontend IDs for builtins, externs, types, constructors, declarations, parameters, pattern bindings, and locals; only compiler-synthesized nodes allocate IDs above the frontend range. Named record and union references remain nominal in HIR and LIR; their declaration shapes are carried separately and serialized in artifacts. Export metadata retains the exported source name and span through LIR and artifacts while the legacy Boolean remains available to compatibility consumers. Conversion evidence is produced by inference, not rediscovered from backend representations. HIR contains no Go types. -LIR is executable but not a Go AST. Lambda bodies are lifted into a deterministic program-level function table. Closure creation references a function ID and carries its canonical ordered set of captured symbol IDs, alongside resolved primitive operations, aggregate construction/projection, and structured control flow. Validation recomputes each lifted function's free-symbol set and rejects unknown function IDs or forged, missing, duplicate, or unordered captures. Pattern matching becomes deterministic tests and projections before reaching a backend. Semantic types remain alongside selected runtime representations for validation and diagnostics. +LIR is executable but not a Go AST. Lambda bodies are lifted into a deterministic program-level function table. Closure creation references a function ID and carries its canonical ordered set of captured symbol IDs, alongside resolved primitive operations, aggregate construction/projection, and structured control flow. Validation recomputes each lifted function's free-symbol set and rejects unknown function IDs or forged, missing, duplicate, or unordered captures. Pattern matching becomes an ordered decision tree with typed, serialized pattern plans before reaching a backend. Semantic types remain alongside selected runtime representations for validation and diagnostics. The Go backend selects concrete generated structs for ordinary named records, with deterministic synthetic field identifiers and FScript-name tags. Declared unions use generated tagged structs with a field for each payload-bearing case and implement a small runtime conversion interface for generic matching and formatting. Structural records remain runtime maps, and explicit LIR conversion nodes copy fields between the representations while evaluating their source once. Runtime record projection, patterns, formatting, and equality reflect only tagged generated fields, so nominal representation does not alter FScript structural behavior. Imported record adapters remain isolated at target-call boundaries. @@ -38,7 +38,7 @@ HIR validation rejects duplicate and unresolved symbols, invalid declaration spa Lexical `let` is retained as a structured LIR node. Its symbol is visible only in the body, or in both the binding and body for recursive functions. The interpreter closes the recursive environment explicitly; Go emission declares the function slot before assigning its closure. `raise` becomes an explicit `Fail` node so consumers preserve evaluation order and attach their own stage-specific failure diagnostic. -Matches lower to a structured decision tree whose target is evaluated once. Cases remain ordered; each pattern test creates artifact-local bindings, then its guard is evaluated, and only then is its body selected. A failed guard continues at the next case. Exhaustion is the explicit FScript match-failure path. Literal, list, tuple, record, and option patterns use the same tree in serialized artifacts, the LIR interpreter, and generated Go. +Matches lower to a structured decision tree whose target is evaluated once. Cases remain ordered; each typed pattern plan creates artifact-local bindings, then its guard is evaluated, and only then is its body selected. A failed guard continues at the next case. Exhaustion is the explicit FScript match-failure path. The Go emitter only serializes the LIR plan and wires its result into bindings and guards; all literal, list, tuple, record, map, option, union, and type tests execute through one runtime plan evaluator rather than backend-specific source expansion. Ordered-map patterns retain ordered clause matching, repeated-binding consistency, remainder binding, and the existing rule that explicit clauses may leave unmatched entries. Type-reference patterns carry their resolved semantic type; consumers test primitive and aggregate representation shape and retain declared union identity. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index c9a0d18..92ce026 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -40,6 +40,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Conditionals and match guards require Boolean values and evaluate only the selected branch. - Match clauses are tried in source order. A guard runs only after its pattern binds successfully. Failure to find a clause is a runtime error. +- Generated Go evaluates each serialized LIR pattern plan through one runtime matcher; the source emitter does not independently expand FScript pattern rules. - Patterns cover literals, variables, wildcards, lists, tuples, records, ordered maps, options, union cases, and type-reference tests. Let-pattern failure is a runtime error. - Compiled map patterns consume entries deterministically in map order, merge repeated bindings only when structurally equal, and bind an optional tail to the remaining ordered map. Compiled type-reference patterns use resolved types rather than source text lookup. - Record updates evaluate the base once and then updates in source order. Ranges, loops, interpolation, indexing, raise, append, and cons preserve the evaluator's existing evaluation order and failures. diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index 18f6124..28972ab 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -166,53 +166,13 @@ module GoBackend = $"fsruntime.Pattern{{Kind: \"union\", TypeName: {quoted typeName}, CaseName: {quoted caseName}, Items: {payloadValue}}}" | Lir.TypePattern patternType -> $"fsruntime.Pattern{{Kind: \"type\", TypeName: {quoted (Types.typeToString patternType)}}}" - let rec private emitPattern valueCode (pattern: Lir.Pattern) = - let combine parts = - let tests, assignments = List.unzip parts - let test = tests |> List.filter ((<>) "true") |> function [] -> "true" | values -> String.concat " && " values - test, List.concat assignments - match pattern.Node with - | Lir.Wildcard -> "true", [] - | Lir.Binding(id, _) -> + let private emitPatternBindings bindingsName (pattern: Lir.Pattern) = + patternBindingDetails pattern + |> List.distinctBy fst + |> List.map (fun (id, bindingType) -> let name = identifier "match" id - "true", [ $"{name} := {cast pattern.Type valueCode}; _ = {name}" ] - | Lir.LiteralPattern value -> $"fsruntime.Equal({valueCode}, {constant value})", [] - | Lir.EmptyList -> $"len(fsruntime.List({valueCode})) == 0", [] - | Lir.NonePattern -> $"!fsruntime.IsSome({valueCode})", [] - | Lir.SomePattern item -> - let test, assignments = emitPattern $"fsruntime.OptionValue({valueCode})" item - $"fsruntime.IsSome({valueCode}) && ({test})", assignments - | Lir.ConsPattern(head, tail) -> - let headPart = emitPattern $"fsruntime.List({valueCode})[0]" head - let tailPart = emitPattern $"fsruntime.Tail({valueCode})" tail - let test, assignments = combine [ headPart; tailPart ] - $"len(fsruntime.List({valueCode})) > 0 && ({test})", assignments - | Lir.TuplePattern items -> - items |> List.mapi (fun index item -> emitPattern $"fsruntime.AsTuple({valueCode})[{index}]" item) - |> combine |> fun (test, assignments) -> $"len(fsruntime.AsTuple({valueCode})) == {items.Length} && ({test})", assignments - | Lir.RecordPattern fields -> - fields |> List.map (fun (name, item) -> - let quoted = System.Text.Json.JsonSerializer.Serialize(name) - let test, assignments = emitPattern $"fsruntime.Record({valueCode})[{quoted}]" item - $"fsruntime.HasField({valueCode}, {quoted}) && ({test})", assignments) - |> combine - | Lir.MapPattern _ -> - let descriptor = emitPatternDescriptor pattern - let assignments = - patternBindingDetails pattern |> List.distinctBy fst |> List.map (fun (id, bindingType) -> - let name = identifier "match" id - let bindingValue = $"fsruntime.PatternBinding({valueCode}, {descriptor}, {Lir.symbolNumber id})" - $"{name} := {cast bindingType bindingValue}; _ = {name}") - $"fsruntime.PatternMatches({valueCode}, {descriptor})", assignments - | Lir.UnionPattern(typeName, caseName, payload) -> - let quotedType = System.Text.Json.JsonSerializer.Serialize(typeName) - let quotedCase = System.Text.Json.JsonSerializer.Serialize(caseName) - match payload with - | None -> $"fsruntime.IsUnion({valueCode}, {quotedType}, {quotedCase}) && !fsruntime.UnionHasValue({valueCode})", [] - | Some item -> - let test, assignments = emitPattern $"fsruntime.UnionValue({valueCode})" item - $"fsruntime.IsUnion({valueCode}, {quotedType}, {quotedCase}) && fsruntime.UnionHasValue({valueCode}) && ({test})", assignments - | Lir.TypePattern patternType -> $"fsruntime.TypeMatches({valueCode}, {System.Text.Json.JsonSerializer.Serialize(Types.typeToString patternType)})", [] + let bindingValue = $"fsruntime.PatternBinding({bindingsName}, {Lir.symbolNumber id})" + $"{name} := {cast bindingType bindingValue}; _ = {name}") let private emitIntrinsic name intrinsicType = let quoted = System.Text.Json.JsonSerializer.Serialize(name) @@ -371,13 +331,16 @@ module GoBackend = emitExpression environment target |> Result.bind (fun targetCode -> let mutable failure = None let clauses = StringBuilder() - for case in cases do + for caseIndex, case in cases |> List.indexed do let bindings = patternBindings case.Pattern let caseEnvironment = bindings |> List.fold (fun env id -> Map.add id (identifier "match" id) env) environment match case.Guard |> Option.map (emitExpression caseEnvironment) |> Option.defaultValue (Ok "true"), emitExpression caseEnvironment case.Body with | Ok guardCode, Ok bodyCode -> - let test, assignments = emitPattern "matchValue" case.Pattern - clauses.Append($"\tif {test} {{\n") |> ignore + let descriptor = emitPatternDescriptor case.Pattern + let bindingsName = $"patternBindings{caseIndex}" + let matchedName = $"patternMatched{caseIndex}" + let assignments = emitPatternBindings bindingsName case.Pattern + clauses.Append($"\t{bindingsName}, {matchedName} := fsruntime.MatchPattern(matchValue, {descriptor})\n\t_ = {bindingsName}\n\tif {matchedName} {{\n") |> ignore assignments |> List.iter (fun assignment -> clauses.Append("\t\t").Append(assignment.Replace("; ", "\n\t\t")).Append("\n") |> ignore) clauses.Append($"\t\tif {guardCode} {{\n\t\t\treturn {indent (indent (indent bodyCode))}\n\t\t}}\n\t}}\n") |> ignore | Error error, _ | _, Error error -> failure <- Some error diff --git a/src/FScript.Backend.Go/Runtime/patterns.go b/src/FScript.Backend.Go/Runtime/patterns.go index 275a366..8612c57 100644 --- a/src/FScript.Backend.Go/Runtime/patterns.go +++ b/src/FScript.Backend.Go/Runtime/patterns.go @@ -172,15 +172,10 @@ func matchRuntimePattern(value any, pattern Pattern) (map[int]any, bool) { } } -func PatternMatches(value any, pattern Pattern) bool { - _, matched := matchRuntimePattern(value, pattern) - return matched +func MatchPattern(value any, pattern Pattern) (map[int]any, bool) { + return matchRuntimePattern(value, pattern) } -func PatternBinding(value any, pattern Pattern, symbol int) any { - bindings, matched := matchRuntimePattern(value, pattern) - if !matched { - panic("Pattern did not match") - } +func PatternBinding(bindings map[int]any, symbol int) any { return bindings[symbol] } func TypeMatches(value any, typeName string) bool { diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index 9911864..6238459 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -115,6 +115,18 @@ type GoBackendTests() = Assert.That(program, Does.Contain("Case0 int64")) Assert.That(program, Does.Contain("FScriptUnion() fsruntime.Union")) + [] + member _.``Go emitter delegates LIR pattern plans to the runtime`` () = + let source = "match Some (1, 2) with | Some (left, right) -> left + right | None -> 0" + let compiled = match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with Ok value -> value | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + match GoBackend.generate { ModuleName = "fscript.generated/patterns" } compiled.Lir with + | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) + | Ok files -> + let program = files |> List.find (fun file -> file.RelativePath.EndsWith("program.go")) |> _.Content + Assert.That(program, Does.Contain("fsruntime.MatchPattern(matchValue")) + Assert.That(program, Does.Not.Contain("fsruntime.OptionValue(matchValue")) + Assert.That(program, Does.Not.Contain("fsruntime.AsTuple(matchValue")) + [] [] [] From a2eea801661d6fe7af52e7e52e23b63f91435dfb Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 21:07:13 +0200 Subject: [PATCH 054/133] fix(compiler): short-circuit LIR execution failures --- CHANGELOG.md | 2 + docs/architecture/assemblies-and-roles.md | 1 + docs/architecture/typed-ir-and-backends.md | 2 +- docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Interpreter.Lir/LirInterpreter.fs | 119 ++++++++++------ .../ConformanceTests.fs | 2 +- .../LirInterpreterTests.fs | 131 +++++++++++------- 7 files changed, 158 insertions(+), 100 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7fcdcd6..ac0faff 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Made LIR execution validate inputs and stop left-to-right evaluation at the first structured diagnostic. + - Routed every generated Go pattern plan through one deterministic runtime matcher. - Preserved arbitrary structured top-level declaration attributes through resolved inference and HIR. diff --git a/docs/architecture/assemblies-and-roles.md b/docs/architecture/assemblies-and-roles.md index 11b521a..dddb24a 100644 --- a/docs/architecture/assemblies-and-roles.md +++ b/docs/architecture/assemblies-and-roles.md @@ -55,6 +55,7 @@ Role: Responsibilities: - Execute LIR values, closures, decision trees, aggregates, intrinsics, and failures. +- Validate caller-provided LIR and return one structured execution diagnostic on failure. - Supply script identity and arguments through `InterpreterContext`. - Reject reachable unsupported target or host linkage with interpreter diagnostics. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index dacff42..36ec0e3 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -34,7 +34,7 @@ HIR declaration schemes are taken directly from the resolved frontend after dete Resolved statements expose normalized structured attributes as name, argument list, and source span. The parser retains unknown attributes instead of assigning semantics to them, and HIR attaches them to ordinary value and type declarations while retaining dedicated export and target-import linkage for consumers that need semantic access without reparsing attribute arguments. -HIR validation rejects duplicate and unresolved symbols, invalid declaration spans, and schemes inconsistent with their values. LIR validation rechecks artifact-local references and spans, then verifies closure calls, branches and decision guards, primitive signatures, aggregate representations, declaration value types, intrinsic catalog membership, and complete target-import linkage. Artifact deserialization runs the same LIR validator, so malformed JSON cannot bypass compiler invariants. +HIR validation rejects duplicate and unresolved symbols, invalid declaration spans, and schemes inconsistent with their values. LIR validation rechecks artifact-local references and spans, then verifies closure calls, branches and decision guards, primitive signatures, aggregate representations, declaration value types, intrinsic catalog membership, and complete target-import linkage. Artifact deserialization and the public LIR execution entry point run the same validator, so malformed or directly constructed LIR cannot bypass compiler invariants. Lexical `let` is retained as a structured LIR node. Its symbol is visible only in the body, or in both the binding and body for recursive functions. The interpreter closes the recursive environment explicitly; Go emission declares the function slot before assigning its closure. `raise` becomes an explicit `Fail` node so consumers preserve evaluation order and attach their own stage-specific failure diagnostic. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 92ce026..3cb3c1d 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -12,6 +12,7 @@ This document records the behavior that the typed IR, LIR interpreter, and compi - Resolved inference exposes parser-independent semantic statement, expression, and pattern trees whose nodes carry their final instantiated type, span, lexical reference kind, resolved primitive operation, and named/structural record-coercion evidence. It also exposes deterministic symbols for every lexical binding category and links each captured reference to its target ID. Repeated occurrences of one pattern binding reuse one ID so equality constraints cannot be lost during lowering. Type-variable numbering in resolved schemes and reference types is normalized independently of prior inference calls. - HIR reuses those resolved IDs unchanged for source symbols even when type and value declarations are interleaved; compiler-synthesized temporaries and constructor payload parameters occupy IDs above the resolved frontend range. - Top-level statements execute in source order. Every non-final expression must have type `unit`; the final expression is the program result. +- LIR expression children execute strictly from left to right and stop at the first failure; later arguments, aggregate fields, interpolation parts, map entries, and operands are not evaluated after that failure. - Compiled top-level expressions execute exactly once in source order even when interleaved with declarations. Top-level pattern bindings evaluate their scrutinee once, fail before later statements when unmatched, and expose all bound names to subsequent statements. The first aggregate compiler slice preserves tuple field order, record field names, option presence, list element order, list cons/append behavior, and recursively structural equality through HIR, serialized LIR, interpretation, and Go generation. diff --git a/src/FScript.Interpreter.Lir/LirInterpreter.fs b/src/FScript.Interpreter.Lir/LirInterpreter.fs index 2544cc5..00f9f3c 100644 --- a/src/FScript.Interpreter.Lir/LirInterpreter.fs +++ b/src/FScript.Interpreter.Lir/LirInterpreter.fs @@ -81,7 +81,7 @@ module LirInterpreter = | VTargetImport _ -> "" let private fail code span message = - Error [ { Code = code; Stage = Interpreter; Message = message; Span = Some span } ] + Error { Code = code; Stage = Interpreter; Message = message; Span = Some span } let private intrinsicArity name = let scheme = @@ -258,10 +258,8 @@ module LirInterpreter = let binary work = match expression.Node with | Lir.Operation(_, [ left; right ]) -> - match evaluate environment left, evaluate environment right with - | Ok a, Ok b -> work a b - | Error a, Error b -> Error(a @ b) - | Error errors, _ | _, Error errors -> Error errors + evaluate environment left + |> Result.bind (fun leftValue -> evaluate environment right |> Result.bind (work leftValue)) | _ -> fail "FSI3002" expression.Span "Invalid primitive arity." match expression.Node with | Lir.Constant value -> @@ -276,10 +274,8 @@ module LirInterpreter = |> Option.map (fun fn -> Ok(VClosure(fn.Parameter, fn.Name, fn.Body, ref captured))) |> Option.defaultWith (fun () -> fail "FSI3024" expression.Span $"Unknown lifted LIR function {Lir.functionNumber functionId}.") | Lir.CallClosure(fn, argument) -> - match evaluate environment fn, evaluate environment argument with - | Ok fnValue, Ok value -> applyValue expression.Span fnValue value - | Error a, Error b -> Error(a @ b) - | Error errors, _ | _, Error errors -> Error errors + evaluate environment fn + |> Result.bind (fun fnValue -> evaluate environment argument |> Result.bind (applyValue expression.Span fnValue)) | Lir.Branch(condition, whenTrue, whenFalse) -> match evaluate environment condition with | Ok(VBool true) -> evaluate environment whenTrue @@ -311,8 +307,7 @@ module LirInterpreter = | Lir.Tuple items -> evaluateMany environment items |> Result.map VTuple | Lir.List items -> evaluateMany environment items |> Result.map VList | Lir.Record fields -> - fields |> List.map (fun (name, item) -> evaluate environment item |> Result.map (fun value -> name, value)) - |> collect |> Result.map (Map.ofList >> VRecord) + evaluateNamed environment fields |> Result.map (Map.ofList >> VRecord) | Lir.FieldGet(target, name) -> match evaluate environment target with | Ok(VRecord fields) -> fields |> Map.tryFind name |> Option.map Ok |> Option.defaultWith (fun () -> fail "FSI3007" expression.Span $"Record field '{name}' was not found.") @@ -321,13 +316,18 @@ module LirInterpreter = | Lir.SomeValue item -> evaluate environment item |> Result.map (Some >> VOption) | Lir.NoneValue -> Ok(VOption None) | Lir.Cons(head, tail) -> - match evaluate environment head, evaluate environment tail with - | Ok h, Ok(VList t) -> Ok(VList(h :: t)) | Ok _, Ok _ -> fail "FSI3008" expression.Span "Cons tail must be a list." - | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e + evaluate environment head + |> Result.bind (fun headValue -> + evaluate environment tail + |> Result.bind (function VList tailValues -> Ok(VList(headValue :: tailValues)) | _ -> fail "FSI3008" expression.Span "Cons tail must be a list.")) | Lir.Append(left, right) -> - match evaluate environment left, evaluate environment right with - | Ok(VList l), Ok(VList r) -> Ok(VList(l @ r)) | Ok _, Ok _ -> fail "FSI3009" expression.Span "Append operands must be lists." - | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e + evaluate environment left + |> Result.bind (fun leftValue -> + evaluate environment right + |> Result.bind (fun rightValue -> + match leftValue, rightValue with + | VList leftValues, VList rightValues -> Ok(VList(leftValues @ rightValues)) + | _ -> fail "FSI3009" expression.Span "Append operands must be lists.")) | Lir.Let(id, _, recursive, binding, body) -> if recursive then match evaluate environment binding with @@ -341,8 +341,8 @@ module LirInterpreter = evaluate environment binding |> Result.bind (fun value -> evaluate (environment |> Map.add id value) body) | Lir.LetRecursiveGroup(bindings, body) -> let evaluated = bindings |> List.map (fun (id, _, _, binding) -> id, evaluate environment binding) - let errors = evaluated |> List.collect (fun (_, result) -> match result with Error values -> values | Ok _ -> []) - if not errors.IsEmpty then Error errors + let error = evaluated |> List.tryPick (fun (_, result) -> match result with Error value -> Some value | Ok _ -> None) + if error.IsSome then Error error.Value elif evaluated |> List.exists (fun (_, result) -> match result with Ok(VClosure _) -> false | _ -> true) then fail "FSI3010" expression.Span "Recursive local groups must contain only functions." else @@ -374,17 +374,20 @@ module LirInterpreter = | Error errors -> Error errors run cases) | Lir.Range(startValue, endValue) -> - match evaluate environment startValue, evaluate environment endValue with - | Ok(VInt first), Ok(VInt last) -> - let step = if first <= last then 1L else -1L - Ok(VList([ first .. step .. last ] |> List.map VInt)) - | Ok _, Ok _ -> fail "FSI3014" expression.Span "Range endpoints must be integers." - | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e + evaluate environment startValue + |> Result.bind (fun firstValue -> + evaluate environment endValue + |> Result.bind (fun lastValue -> + match firstValue, lastValue with + | VInt first, VInt last -> + let step = if first <= last then 1L else -1L + Ok(VList([ first .. step .. last ] |> List.map VInt)) + | _ -> fail "FSI3014" expression.Span "Range endpoints must be integers.")) | Lir.RecordUpdate(target, fields) -> match evaluate environment target with | Ok(VRecord existing) -> - fields |> List.map (fun (name, item) -> evaluate environment item |> Result.map (fun value -> name, value)) - |> collect |> Result.map (fun updates -> VRecord(List.fold (fun state (name, value) -> Map.add name value state) existing updates)) + evaluateNamed environment fields + |> Result.map (fun updates -> VRecord(List.fold (fun state (name, value) -> Map.add name value state) existing updates)) | Ok _ -> fail "FSI3015" expression.Span "Record update requires a record." | Error errors -> Error errors | Lir.RecordConvert(target, _) -> @@ -393,16 +396,18 @@ module LirInterpreter = | Ok _ -> fail "FSI3015" expression.Span "Record conversion requires a record." | Error errors -> Error errors | Lir.IndexGet(target, index) -> - match evaluate environment target, evaluate environment index with - | Ok(VList values), Ok(VInt position) -> - if position < 0L || position >= int64 values.Length then Ok(VOption None) - else Ok(VOption(Some values.[int position])) - | Ok(VMap entries), Ok key -> Ok(VOption(entries |> List.tryPick (fun (itemKey, value) -> if equal itemKey key then Some value else None))) - | Ok _, Ok _ -> fail "FSI3016" expression.Span "Index access requires a list and integer index." - | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e + evaluate environment target + |> Result.bind (fun targetValue -> + evaluate environment index + |> Result.bind (fun indexValue -> + match targetValue, indexValue with + | VList values, VInt position -> + if position < 0L || position >= int64 values.Length then Ok(VOption None) + else Ok(VOption(Some values.[int position])) + | VMap entries, key -> Ok(VOption(entries |> List.tryPick (fun (itemKey, value) -> if equal itemKey key then Some value else None))) + | _ -> fail "FSI3016" expression.Span "Index access requires a list and integer index.")) | Lir.Interpolate parts -> - parts |> List.map (function Lir.TextPart text -> Ok text | Lir.ValuePart item -> evaluate environment item |> Result.map formatValue) - |> collect |> Result.map (String.concat "" >> VString) + evaluateInterpolation environment parts |> Result.map (String.concat "" >> VString) | Lir.UnionCase(typeName, caseName, payload) -> payload |> Option.map (fun value -> evaluate environment value |> Result.map Some) |> Option.defaultValue (Ok None) |> Result.map (fun value -> VUnion(typeName, caseName, value)) @@ -414,9 +419,10 @@ module LirInterpreter = match remaining with | [] -> Ok(VMap values) | Lir.KeyValue(key, item) :: rest -> - match evaluate environment key, evaluate environment item with - | Ok keyValue, Ok itemValue -> build (add keyValue itemValue values) rest - | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e + evaluate environment key + |> Result.bind (fun keyValue -> + evaluate environment item + |> Result.bind (fun itemValue -> build (add keyValue itemValue values) rest)) | Lir.Spread item :: rest -> match evaluate environment item with | Ok(VMap spread) -> @@ -460,13 +466,29 @@ module LirInterpreter = else Ok(VTargetImport(modulePath, memberName, arity, arguments)) | _ -> fail "FSI3003" span "Attempted to call a non-function value." - and private evaluateMany environment items = items |> List.map (evaluate environment) |> collect + and private evaluateMany environment items = + match items with + | [] -> Ok [] + | item :: rest -> + evaluate environment item + |> Result.bind (fun value -> evaluateMany environment rest |> Result.map (fun values -> value :: values)) - and private collect<'a> (results: Result<'a, CompilerDiagnostic list> list) : Result<'a list, CompilerDiagnostic list> = - results |> List.fold (fun state result -> - match state, result with Ok values, Ok value -> Ok(values @ [ value ]) | Error a, Error b -> Error(a @ b) | Error e, _ | _, Error e -> Error e) (Ok []) + and private evaluateNamed environment fields = + match fields with + | [] -> Ok [] + | (name, item) :: rest -> + evaluate environment item + |> Result.bind (fun value -> evaluateNamed environment rest |> Result.map (fun values -> (name, value) :: values)) - let execute (context: InterpreterContext) (program: Lir.Program) = + and private evaluateInterpolation environment parts = + match parts with + | [] -> Ok [] + | Lir.TextPart text :: rest -> evaluateInterpolation environment rest |> Result.map (fun values -> text :: values) + | Lir.ValuePart item :: rest -> + evaluate environment item + |> Result.bind (fun value -> evaluateInterpolation environment rest |> Result.map (fun values -> formatValue value :: values)) + + let private executeValidated (context: InterpreterContext) (program: Lir.Program) = let output = ResizeArray() currentOutput.Value <- output currentFunctions.Value <- program.Functions |> List.map (fun fn -> fn.Id, fn) |> Map.ofList @@ -490,8 +512,8 @@ module LirInterpreter = let group = declarations |> List.takeWhile _.IsRecursive let remainder = declarations |> List.skip group.Length let closures = group |> List.map (fun declaration -> declaration, evaluate environment.Value declaration.Value) - let errors = closures |> List.collect (fun (_, result) -> match result with Error values -> values | Ok _ -> []) - if not errors.IsEmpty then failure <- Some errors + let error = closures |> List.tryPick (fun (_, result) -> match result with Error value -> Some value | Ok _ -> None) + if error.IsSome then failure <- error else let next = closures |> List.fold (fun env (declaration, result) -> match result with Ok closure -> Map.add declaration.Id closure env | Error _ -> env) environment.Value for _, result in closures do @@ -503,3 +525,8 @@ module LirInterpreter = | Some errors, _ -> Error errors | None, Some entry -> evaluate environment.Value entry |> Result.map (fun value -> { Value = value; Stdout = List.ofSeq output }) | None, None -> Ok { Value = VUnit; Stdout = List.ofSeq output } + + let execute (context: InterpreterContext) (program: Lir.Program) : Result = + match LirValidation.validate program with + | Error diagnostics -> Error diagnostics.Head + | Ok validated -> executeValidated context validated diff --git a/tests/FScript.Conformance.Tests/ConformanceTests.fs b/tests/FScript.Conformance.Tests/ConformanceTests.fs index 982aa47..41e71d4 100644 --- a/tests/FScript.Conformance.Tests/ConformanceTests.fs +++ b/tests/FScript.Conformance.Tests/ConformanceTests.fs @@ -51,7 +51,7 @@ module private Harness = let loaded = match IrArtifact.deserialize artifact with Ok value -> value | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) match LirInterpreter.execute { ScriptName = None; Arguments = [] } loaded with | Ok result -> Ok { Value = formatLir result.Value; Stdout = result.Stdout } - | Error diagnostics -> Error diagnostics.Head.Code + | Error diagnostic -> Error diagnostic.Code let go source = let files = match GoBackend.generate { ModuleName = "fscript.generated/conformance" } (compile source).Lir with Ok value -> value | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) diff --git a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs index a1394ff..c6d6237 100644 --- a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs +++ b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs @@ -6,13 +6,19 @@ open NUnit.Framework [] type LirInterpreterTests() = + let errorText (value: obj) = + match value with + | :? CompilerDiagnostic as diagnostic -> diagnostic.Message + | :? (CompilerDiagnostic list) as diagnostics -> System.String.Join("; ", diagnostics |> List.map _.Message) + | _ -> string value + [] member _.``Captures portable console output`` () = match Compiler.compile { Program = Parser.parseProgram "Console.writeLine \"hello\""; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VUnit)) Assert.That(System.String.Join("\n", result.Stdout), Is.EqualTo("hello")) @@ -22,43 +28,43 @@ type LirInterpreterTests() = member _.``Executes aggregate expressions in LIR`` () = let input = { Program = Parser.parseProgram "[1] @ [2; 3]"; Externs = [] } match Compiler.compile input with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VList [ LirValue.VInt 1L; LirValue.VInt 2L; LirValue.VInt 3L ])) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) [] [] [] member _.``Lowers tuples options and records`` (source: string) = match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok _ -> Assert.Pass() - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) [] member _.``Executes lexical let bindings and recursive closures`` () = let source = "let f x = (let rec loop n = if n = 0 then x else loop (n - 1)\n loop 2\n)\nf 7" match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 7L)) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) [] member _.``Raise preserves an interpreter diagnostic`` () = match Compiler.compile { Program = Parser.parseProgram "raise \"boom\""; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok _ -> Assert.Fail("Expected failure") | Error diagnostics -> - Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSI3011")) - Assert.That(diagnostics |> List.map _.Message, Does.Contain("boom")) + Assert.That(diagnostics.Code, Is.EqualTo("FSI3011")) + Assert.That(diagnostics.Message, Is.EqualTo("boom")) [ x\n | [] -> 0", 1L)>] [ x + 1\n | None -> 0", 4L)>] @@ -66,11 +72,11 @@ type LirInterpreterTests() = [ x\n | _ -> 42", 42L)>] member _.``Executes deterministic pattern decision trees`` (source: string, expected: int64) = match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt expected)) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) [] member _.``Executes ranges indexing updates and interpolation`` () = @@ -81,41 +87,41 @@ type LirInterpreterTests() = "$\"value {1 + 2}\"", LirValue.VString "value 3" ] for source, expected in cases do match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(expected), source) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) [] member _.``Executes declared union construction and matching`` () = let source = "type Shape = | Point | Circle of int\nlet radius shape =\n match shape with\n | Point -> 0\n | Circle r -> r\nradius (Circle 3)" match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 3L)) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) [] member _.``Executes ordered map construction spread and indexing`` () = let source = "let tail = { [\"a\"] = 2; [\"b\"] = 3 }\nlet values = { [\"a\"] = 1; ..tail }\nvalues[\"b\"]" match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VOption(Some(LirValue.VInt 3L)))) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) [] member _.``Initializes mutual recursive groups atomically`` () = let source = "let rec even n =\n if n = 0 then true else odd (n - 1)\nand odd n =\n if n = 0 then false else even (n - 1)\n\neven 10" match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VBool true)) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) [] [] @@ -123,11 +129,11 @@ type LirInterpreterTests() = [] member _.``Executes portable scalar and collection intrinsics`` (source: string, expected: string) = match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VString expected)) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) [] member _.``Retains host linkage but rejects only reachable external calls`` () = @@ -138,49 +144,49 @@ type LirInterpreterTests() = Kind = UnsupportedHostExtern } let compile source = Compiler.compile { Program = Parser.parseProgram source; Externs = [ external ] } match compile "1" with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 1L)) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) match compile "Host.echo \"value\"" with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok _ -> Assert.Fail("Expected unsupported external diagnostic") - | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSI3021")) + | Error diagnostics -> Assert.That(diagnostics.Code, Is.EqualTo("FSI3021")) [] member _.``Default host runtime externs remain inert until reached`` () = let execute source = match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Error diagnostics + | Error diagnostics -> Error diagnostics.Head | Ok compiled -> LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir match execute "let unused = Fs.exists\n42" with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 42L)) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) match execute "Fs.exists \"missing\"" with | Ok _ -> Assert.Fail("Expected unsupported host-runtime diagnostic") - | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSI3021")) + | Error diagnostics -> Assert.That(diagnostics.Code, Is.EqualTo("FSI3021")) [] member _.``Retains tasks in LIR and rejects them only when reached`` () = let compile source = Compiler.compile { Program = Parser.parseProgram source; Externs = [] } match compile "let unused = Task.spawn\n1" with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 1L)) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) match compile "Task.await (Task.spawn (fun _ -> 42))" with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> Assert.That(compiled.Lir.Declarations |> List.exists (fun declaration -> match declaration.Value.Node with Lir.HostExternal("Task.spawn", 1) -> true | _ -> false), Is.True) match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok _ -> Assert.Fail("Expected unsupported task diagnostic") - | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSI3021")) + | Error diagnostics -> Assert.That(diagnostics.Code, Is.EqualTo("FSI3021")) [] member _.``Executes local binding patterns iteration and type tokens`` () = @@ -190,11 +196,11 @@ type LirInterpreterTests() = "type Package = { Name: string }\ntypeof Package", LirValue.VTypeToken(TRecord(Map.ofList [ "Name", TString ])) ] for source, expected in cases do match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(expected), source) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) [] member _.``Executes the complete portable collection intrinsic catalog`` () = @@ -210,71 +216,92 @@ type LirInterpreterTests() = "Map.fold (fun state key value -> state + value) 0 { [\"a\"] = 1; [\"b\"] = 2 }", LirValue.VInt 3L ] for source, expected in cases do match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(expected), source) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) [] member _.``Preserves top-level pattern bindings and expression order`` () = match Compiler.compile { Program = Parser.parseProgram "let (left, right) = (1, 2)\nleft + right"; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 3L)) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) match Compiler.compile { Program = Parser.parseProgram "raise \"first\"\nraise \"second\""; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok _ -> Assert.Fail("Expected the first top-level expression to fail") - | Error diagnostics -> Assert.That(diagnostics.Head.Message, Is.EqualTo("first")) + | Error diagnostics -> Assert.That(diagnostics.Message, Is.EqualTo("first")) + + [] + member _.``Stops evaluating expression children after the first failure`` () = + let source = "(raise \"first\", raise \"second\")" + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(errorText diagnostics) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok _ -> Assert.Fail("Expected tuple evaluation to fail") + | Error diagnostic -> Assert.That(diagnostic.Message, Is.EqualTo("first")) + + [] + member _.``Validates LIR before execution`` () = + match Compiler.compile { Program = Parser.parseProgram "1"; Externs = [] } with + | Error diagnostics -> Assert.Fail(errorText diagnostics) + | Ok compiled -> + let entry = compiled.Lir.Entry.Value + let invalid = { compiled.Lir with Entry = Some { entry with Node = Lir.Load(Lir.SymbolId 999999) } } + match LirInterpreter.execute { ScriptName = None; Arguments = [] } invalid with + | Ok _ -> Assert.Fail("Expected invalid LIR to be rejected") + | Error diagnostic -> Assert.That(diagnostic.Code, Does.StartWith("FSL2")) [] member _.``Executes local mutual recursive closure groups`` () = let source = "(let rec even n = if n = 0 then true else odd (n - 1)\nand odd n = if n = 0 then false else even (n - 1)\neven 10\n)" match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VBool true)) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) [ (key, value, Map.count tail)\n | {} -> (\"\", 0, 0)")>] [ (\"Name\", 1, 0)\n | _ -> (\"\", 0, 0)")>] member _.``Executes map and type-reference patterns`` (source: string) = match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok _ -> Assert.Pass() - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) [] member _.``Supplies Env from the LIR interpreter context`` () = let source = "(Env.ScriptName, Env.Arguments)" match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = Some "sample.fss"; Arguments = [ "one"; "two" ] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VTuple [ LirValue.VOption(Some(LirValue.VString "sample.fss")); LirValue.VList [ LirValue.VString "one"; LirValue.VString "two" ] ])) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) [] member _.``Rejects only reachable target imports`` () = let compile source = Compiler.compile { Program = Parser.parseProgram source; Externs = [] } match compile "[] let upper : string -> string\n1" with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VInt 1L)) - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) match compile "[] let upper : string -> string\nupper \"hello\"" with - | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) + | Error diagnostics -> Assert.Fail(errorText diagnostics) | Ok compiled -> match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with | Ok _ -> Assert.Fail("Expected target-import diagnostic") - | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSI3023")) + | Error diagnostics -> Assert.That(diagnostics.Code, Is.EqualTo("FSI3023")) From 573699b24e2c117c4535a489d5a7eb1036d65d53 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 21:10:19 +0200 Subject: [PATCH 055/133] fix(go): validate LIR before generation --- CHANGELOG.md | 2 ++ docs/architecture/typed-ir-and-backends.md | 2 +- src/FScript.Backend.Go/GoBackend.fs | 5 ++++- tests/FScript.Backend.Go.Tests/GoBackendTests.fs | 9 +++++++++ 4 files changed, 16 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ac0faff..c5a811c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Made Go generation reject invalid caller-provided LIR before representation selection. + - Made LIR execution validate inputs and stop left-to-right evaluation at the first structured diagnostic. - Routed every generated Go pattern plan through one deterministic runtime matcher. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 36ec0e3..0e4c01e 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -34,7 +34,7 @@ HIR declaration schemes are taken directly from the resolved frontend after dete Resolved statements expose normalized structured attributes as name, argument list, and source span. The parser retains unknown attributes instead of assigning semantics to them, and HIR attaches them to ordinary value and type declarations while retaining dedicated export and target-import linkage for consumers that need semantic access without reparsing attribute arguments. -HIR validation rejects duplicate and unresolved symbols, invalid declaration spans, and schemes inconsistent with their values. LIR validation rechecks artifact-local references and spans, then verifies closure calls, branches and decision guards, primitive signatures, aggregate representations, declaration value types, intrinsic catalog membership, and complete target-import linkage. Artifact deserialization and the public LIR execution entry point run the same validator, so malformed or directly constructed LIR cannot bypass compiler invariants. +HIR validation rejects duplicate and unresolved symbols, invalid declaration spans, and schemes inconsistent with their values. LIR validation rechecks artifact-local references and spans, then verifies closure calls, branches and decision guards, primitive signatures, aggregate representations, declaration value types, intrinsic catalog membership, and complete target-import linkage. Artifact deserialization, LIR execution, and Go generation run the same validator, so malformed or directly constructed LIR cannot bypass compiler invariants. Lexical `let` is retained as a structured LIR node. Its symbol is visible only in the body, or in both the binding and body for recursive functions. The interpreter closes the recursive environment explicitly; Go emission declares the function slot before assigning its closure. `raise` becomes an explicit `Fail` node so consumers preserve evaluation order and attach their own stage-specific failure diagnostic. diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index 28972ab..2f1afcb 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -643,7 +643,7 @@ func Environment() map[string]any { with ex -> Error [ { Code = "FSB4002"; Stage = GoGeneration; Message = ex.Message; Span = None } ] - let generate options (program: Lir.Program) : Result = + let private generateValidated options (program: Lir.Program) : Result = currentImportedRecords.Value <- Some(program.ImportedRecords |> List.map (fun record -> record.Name, record) |> Map.ofList) currentTypes.Value <- Some(program.Types |> List.map (fun declaration -> declaration.Name, declaration) |> Map.ofList) currentFunctions.Value <- program.Functions |> List.map (fun fn -> fn.Id, fn) |> Map.ofList @@ -710,3 +710,6 @@ func Environment() map[string]any { | None, Some span, _ -> Error [ { Code = "FSB4004"; Stage = GoGeneration; Message = "Task values are not supported by the Go backend."; Span = Some span } ] | None, None, Some(name, span) -> Error [ { Code = "FSB4006"; Stage = GoGeneration; Message = $"Host external '{name}' is not supported by the Go backend."; Span = Some span } ] | None, None, None -> generateCore options program + + let generate options (program: Lir.Program) : Result = + program |> LirValidation.validate |> Result.bind (generateValidated options) diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index 6238459..f096f66 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -32,6 +32,15 @@ type GoBackendTests() = | Ok _ -> Assert.Fail("Expected unsupported task diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSB4004")) + [] + member _.``Go backend validates caller-provided LIR`` () = + let compiled = match Compiler.compile { Program = Parser.parseProgram "1"; Externs = [] } with Ok value -> value | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + let entry = compiled.Lir.Entry.Value + let invalid = { compiled.Lir with Entry = Some { entry with Node = Lir.Load(Lir.SymbolId 999999) } } + match GoBackend.generate { ModuleName = "fscript.generated/invalid" } invalid with + | Ok _ -> Assert.Fail("Expected invalid LIR to be rejected") + | Error diagnostics -> Assert.That(diagnostics |> List.exists (fun diagnostic -> diagnostic.Code.StartsWith("FSL2")), Is.True) + [] member _.``Go backend rejects only reachable host externs`` () = let external = From bb489eef62a2cca6d636ee701989c0735c2cac2f Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 21:15:04 +0200 Subject: [PATCH 056/133] test(compiler): harden artifact and Go gates --- CHANGELOG.md | 2 ++ docs/architecture/typed-ir-and-backends.md | 2 +- src/FScript.Compiler/IrArtifact.fs | 24 ++++++++++++++----- .../GoBackendTests.fs | 14 +++++++++++ .../FScript.Compiler.Tests/IrArtifactTests.fs | 15 ++++++++++++ 5 files changed, 50 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c5a811c..4ce28c5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Hardened canonical IR metadata validation and required generated projects to pass `go test ./...`. + - Made Go generation reject invalid caller-provided LIR before representation selection. - Made LIR execution validate inputs and stop left-to-right evaluation at the first structured diagnostic. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 0e4c01e..541b4c9 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -64,7 +64,7 @@ Every top-level source expression becomes a synthetic declaration at its exact p Serialized `.fsir` files contain one closed resolved program plus explicit foreign linkage declarations. JSON format versions are internal and may be rejected rather than upgraded. -Artifact metadata records the actual compiler and language assembly versions, normalized paths and SHA-256 hashes for available file-backed sources, and a sorted unique list of foreign dependencies. Deserialization rejects missing metadata, malformed hashes, non-canonical dependency order, unknown formats or versions, and any LIR that fails post-load validation. +Artifact metadata records the actual compiler and language assembly versions, normalized paths and lowercase SHA-256 hashes for available file-backed sources, and a sorted unique list of foreign dependencies. Deserialization rejects empty versions, duplicate or unsorted sources/dependencies, non-normalized paths, malformed hashes, missing metadata, unknown formats or versions, and any LIR that fails post-load validation. ## Representation policy diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index 44bbe95..60513b4 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -449,16 +449,28 @@ module IrArtifact = elif [ "compilerVersion"; "languageVersion"; "sources"; "dependencies"; "program" ] |> List.exists (fun name -> not (hasProperty name root)) then Error [ { Code = "FSA5004"; Stage = Artifact; Message = "IR artifact is missing required metadata."; Span = None } ] else + let sourceElements = + if (property "sources" root).ValueKind = JsonValueKind.Array then + (property "sources" root).EnumerateArray() |> Seq.toList + else [] + let sourceValues = sourceElements |> List.map (fun source -> stringProperty "path" source, stringProperty "sha256" source) + let validHash (hash: string) = + hash.Length = 64 + && hash |> Seq.forall (fun character -> System.Char.IsDigit character || (character >= 'a' && character <= 'f')) let sourcesValid = (property "sources" root).ValueKind = JsonValueKind.Array - && ((property "sources" root).EnumerateArray() |> Seq.forall (fun source -> - hasProperty "path" source && hasProperty "sha256" source - && (stringProperty "sha256" source).Length = 64)) - let dependencyValues = + && sourceElements |> List.forall (fun source -> hasProperty "path" source && hasProperty "sha256" source) + && sourceValues |> List.forall (fun (path, hash) -> not (String.IsNullOrWhiteSpace path) && path = normalizePath path && validHash hash) + && sourceValues = (sourceValues |> List.distinct |> List.sortBy fst) + let dependencyElements = if (property "dependencies" root).ValueKind = JsonValueKind.Array then - (property "dependencies" root).EnumerateArray() |> Seq.map _.GetString() |> Seq.choose Option.ofObj |> Seq.toList + (property "dependencies" root).EnumerateArray() |> Seq.toList else [] - if not sourcesValid || dependencyValues <> (dependencyValues |> List.distinct |> List.sort) then + let dependencyValues = dependencyElements |> List.map _.GetString() |> List.map (Option.ofObj >> Option.defaultValue "") + let versionsValid = + not (String.IsNullOrWhiteSpace(stringProperty "compilerVersion" root)) + && not (String.IsNullOrWhiteSpace(stringProperty "languageVersion" root)) + if not versionsValid || not sourcesValid || dependencyValues |> List.exists String.IsNullOrWhiteSpace || dependencyValues <> (dependencyValues |> List.distinct |> List.sort) then Error [ { Code = "FSA5005"; Stage = Artifact; Message = "IR artifact metadata is invalid or non-canonical."; Span = None } ] else let programElement = property "program" root diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index f096f66..6db8a1e 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -219,6 +219,20 @@ type GoBackendTests() = formatProcess.WaitForExit() Assert.That(formatProcess.ExitCode, Is.EqualTo(0)) Assert.That(formatDiff, Is.Empty, "Generated Go sources must already be gofmt-compatible") + let testStart = ProcessStartInfo("go") + testStart.WorkingDirectory <- directory + testStart.ArgumentList.Add("test") + testStart.ArgumentList.Add("./...") + testStart.RedirectStandardOutput <- true + testStart.RedirectStandardError <- true + use testProcess = + match Process.Start(testStart) with + | null -> failwith "Failed to start go test" + | value -> value + let testOutput = testProcess.StandardOutput.ReadToEnd() + let testError = testProcess.StandardError.ReadToEnd() + testProcess.WaitForExit() + Assert.That(testProcess.ExitCode, Is.EqualTo(0), testOutput + testError) let start = ProcessStartInfo("go") start.WorkingDirectory <- directory start.ArgumentList.Add("run") diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index f9160a2..ca0c022 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -62,6 +62,21 @@ type IrArtifactTests() = | Ok _ -> Assert.Fail("Expected non-canonical-metadata diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSA5005")) + let invalidSource = parseObject bytes + let sourceMetadata = JsonObject() + sourceMetadata["path"] <- "z\\program.fss" + sourceMetadata["sha256"] <- String.replicate 64 "A" + invalidSource["sources"] <- JsonArray(sourceMetadata) + match IrArtifact.deserialize (Encoding.UTF8.GetBytes(invalidSource.ToJsonString())) with + | Ok _ -> Assert.Fail("Expected invalid source-metadata diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSA5005")) + + let invalidVersion = parseObject bytes + invalidVersion["compilerVersion"] <- "" + match IrArtifact.deserialize (Encoding.UTF8.GetBytes(invalidVersion.ToJsonString())) with + | Ok _ -> Assert.Fail("Expected invalid version-metadata diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSA5005")) + [] member _.``IR artifact records normalized source paths and SHA256 hashes`` () = let path = IO.Path.Combine(IO.Path.GetTempPath(), $"fscript-source-{Guid.NewGuid():N}.fss") From 6d9c3abcac9f5b7ddc45c760dbb7760b010a5787 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 21:19:45 +0200 Subject: [PATCH 057/133] fix(compiler): preserve ordered map construction --- CHANGELOG.md | 2 ++ docs/architecture/typed-ir-and-backends.md | 6 ++++-- docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Backend.Go/Runtime/runtime.go | 16 ++++++++++++++-- src/FScript.Interpreter.Lir/LirInterpreter.fs | 2 +- tests/FScript.Backend.Go.Tests/GoBackendTests.fs | 1 + .../ConformanceTests.fs | 1 + 7 files changed, 24 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4ce28c5..b96204c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Preserved canonical ordered-map key order across `Map.ofList` and `Map.add` in LIR and Go. + - Hardened canonical IR metadata validation and required generated projects to pass `go test ./...`. - Made Go generation reject invalid caller-provided LIR before representation selection. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 541b4c9..33be666 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -46,13 +46,13 @@ Top-level mutual-recursive declarations are initialized as an atomic closure gro Local `let rec ... and ...` uses the same atomic rule inside a structured expression node. All local names are allocated before any member is lowered, every binding must evaluate to a closure, captures are updated with the complete group, and the body runs only after group initialization. -Language builtins are explicit `Intrinsic` declarations rather than magic unresolved references. Consumers accumulate their curried arguments according to the catalog signature and dispatch only at full application. The initial portable implementations cover scalar parsing/formatting, strings, simple list and option operations, and map queries; a reachable catalog entry without an implementation receives a stage-specific unsupported diagnostic or runtime failure. +Language builtins are explicit `Intrinsic` declarations rather than magic unresolved references. Consumers accumulate their curried arguments according to the catalog signature and dispatch only at full application. Both executable consumers cover the complete pure scalar, string, list, option, and ordered-map catalog; a reachable non-portable linkage receives a stage-specific unsupported diagnostic or runtime failure. The LIR interpreter implements the pure builtin catalog, including callbacks for list mapping, selection, collection, folding, filtering and search; lazy option fallback and mapping; and ordered-map construction, folds, transforms, iteration, and removal. Callback application goes through the same curried closure path as ordinary source calls, preserving failure propagation and evaluation order. Generated Go represents every FScript function as an erased `fsruntime.Closure` with a single `any -> any` call slot. Typed values are checked at closure entry and result use. This uniform convention permits ordinary source closures, partial application, recursion, and higher-order runtime intrinsics to interoperate without Go generics or reflection-based invocation. -The Go runtime is stored as formatted `.go` resources embedded in `FScript.Backend.Go`. Program emission uses multiline structured constructs for closures, local control flow, decision trees, recursive groups, loops, and imports. Generation never invokes Go tooling; the backend tests run `gofmt -d` over every generated project and require an empty diff before `go run`. +The Go runtime is stored as formatted `.go` resources embedded in `FScript.Backend.Go`. Program emission uses multiline structured constructs for closures, local control flow, decision trees, recursive groups, loops, and imports. Generation never invokes Go tooling; the backend tests run `gofmt -d`, `go test ./...`, and `go run` over generated projects. Host externs are copied into IR as name, signature, and arity only; their executable .NET delegates never cross the compiler boundary. Constructing or retaining an unused extern handle is inert. Fully applying one in the LIR interpreter produces `FSI3021`, preserving the rule that unreachable unsupported linkage does not block execution. @@ -99,3 +99,5 @@ The conformance suite evaluates one source program through the legacy evaluator, ## Previous experiment The `feature/in-memory-execution` branch is useful only as evidence about closures, invocation, and host integration. Its dynamic-method/in-memory assembly backend is not an IR layer and is neither copied nor referenced by the new compiler architecture. + +The optional migration of ordinary .NET script execution remains deliberately inactive: task execution and supported host-extern adapters have not reached full parity. The CLI and `ScriptHost` therefore retain the legacy evaluator for execution, and the Fable/JavaScript surface is unchanged; compiler CLI targets, LIR interpretation, and Go generation are available without deleting that reference path. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 3cb3c1d..28a110e 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -22,6 +22,7 @@ Ranges preserve inclusive ascending or descending order. List indexing returns a Declared unions retain their type and case identity plus an optional erased payload. Constructors are deterministic compiler declarations, qualified and unqualified case references share one identity, pattern tests check both type and case tags, and equality recursively compares payloads. Compiled maps use an ordered entry representation for string and integer keys. Explicit key/value entries replace an earlier value; spread entries retain values already present on their left. Indexing returns an option, and deterministic ordering participates in artifacts, equality, formatting, and generated Go behavior. +- `Map.ofList` and `Map.add` restore canonical key order after insertion or replacement rather than exposing source-list insertion order. - Functions are curried. Multi-argument declarations are nested lambdas, application is one argument at a time, and LIR closures record only their deterministic free-symbol capture set. Single and mutual recursive function groups are supported. - Local mutual-recursive function groups initialize atomically: every peer is visible from every closure before the group body executes. diff --git a/src/FScript.Backend.Go/Runtime/runtime.go b/src/FScript.Backend.Go/Runtime/runtime.go index 34a512e..56c7e31 100644 --- a/src/FScript.Backend.Go/Runtime/runtime.go +++ b/src/FScript.Backend.Go/Runtime/runtime.go @@ -233,10 +233,23 @@ func mapSet(entries []MapEntry, key any, value any, replace bool) []MapEntry { if replace { entries[index].Value = value } + sortMapEntries(entries) return entries } } - return append(entries, MapEntry{Key: key, Value: value}) + entries = append(entries, MapEntry{Key: key, Value: value}) + sortMapEntries(entries) + return entries +} +func sortMapEntries(entries []MapEntry) { + sort.SliceStable(entries, func(left int, right int) bool { + leftString, leftIsString := entries[left].Key.(string) + rightString, rightIsString := entries[right].Key.(string) + if leftIsString && rightIsString { + return leftString < rightString + } + return Format(entries[left].Key) < Format(entries[right].Key) + }) } func BuildMap(parts []MapPart) OrderedMap { entries := []MapEntry{} @@ -249,7 +262,6 @@ func BuildMap(parts []MapPart) OrderedMap { entries = mapSet(entries, part.Key, part.Value, true) } } - sort.SliceStable(entries, func(left int, right int) bool { return Format(entries[left].Key) < Format(entries[right].Key) }) return OrderedMap{Entries: entries} } func Index(value any, index any) Option { diff --git a/src/FScript.Interpreter.Lir/LirInterpreter.fs b/src/FScript.Interpreter.Lir/LirInterpreter.fs index 00f9f3c..ed73643 100644 --- a/src/FScript.Interpreter.Lir/LirInterpreter.fs +++ b/src/FScript.Interpreter.Lir/LirInterpreter.fs @@ -109,7 +109,7 @@ module LirInterpreter = | [] -> List.rev ((key, value) :: acc) | (existing, _) :: rest when equal existing key -> List.rev acc @ ((key, value) :: rest) | entry :: rest -> loop (entry :: acc) rest - loop [] entries + loop [] entries |> List.sortBy (fst >> formatValue) match name, arguments with | "Console.writeLine", [ VString value ] -> currentOutput.Value.Add(value); Ok VUnit | "ignore", [ _ ] -> Ok VUnit diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index 6db8a1e..768a1a8 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -171,6 +171,7 @@ type GoBackendTests() = [ value > 1) [1; 2; 3]", "Some 2")>] [] [ 1; \"b\" => 2 }")>] + [ 1; \"b\" => 2 }")>] [ value + 1) (Map.filter (fun key value -> value > 1) (Map.ofList [(\"b\", 2); (\"a\", 1)]))", "map { \"b\" => 3 }")>] [ if value > 1 then Some (value + 1) else None) { [\"a\"] = 1; [\"b\"] = 2 }", "map { \"b\" => 3 }")>] [ 2 }")>] diff --git a/tests/FScript.Conformance.Tests/ConformanceTests.fs b/tests/FScript.Conformance.Tests/ConformanceTests.fs index 41e71d4..a0b2c23 100644 --- a/tests/FScript.Conformance.Tests/ConformanceTests.fs +++ b/tests/FScript.Conformance.Tests/ConformanceTests.fs @@ -120,6 +120,7 @@ type ConformanceTests() = [ value + 1) [1; 2], List.iter (fun value -> ignore value) [1; 2], List.choose (fun value -> if value > 1 then Some value else None) [1; 2], List.collect (fun value -> [value; value]) [1; 2], List.exists (fun value -> value = 2) [1; 2], List.contains 2 [1; 2], List.rev [1; 2], List.distinct [1; 1; 2], List.fold (fun state value -> state + value) 0 [1; 2], List.filter (fun value -> value > 1) [1; 2], List.length [1; 2], List.tryFind (fun value -> value > 1) [1; 2], List.tryGet (fun value -> value = 2) [1; 2], List.tryItem 1 [1; 2], List.tryHead [1; 2], List.tail [1; 2], List.append [1] [2])")>] [ 8) None, Option.isNone None, Option.isSome (Some 1), Option.map (fun value -> value + 1) (Some 1))")>] [ state + value) 0 (Map.ofList [(\"a\", 1); (\"b\", 2)]), Map.filter (fun key value -> value > 1) (Map.ofList [(\"a\", 1); (\"b\", 2)]), Map.choose (fun key value -> if value > 1 then Some value else None) (Map.ofList [(\"a\", 1); (\"b\", 2)]), Map.map (fun value -> value + 1) (Map.ofList [(\"a\", 1)]), Map.iter (fun key value -> ignore value) (Map.ofList [(\"a\", 1)]), Map.remove \"a\" (Map.ofList [(\"a\", 1); (\"b\", 2)]))")>] + [] member _.``Legacy LIR artifact and Go results conform`` (source: string) = let expected = Harness.legacy source match Harness.lir source with From 723365ff3f4c1fd536569b59d6cfba118c41c56d Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 21:24:52 +0200 Subject: [PATCH 058/133] refactor(language): project legacy typed inference --- CHANGELOG.md | 2 ++ docs/architecture/typed-ir-and-backends.md | 2 +- docs/specs/compiler-semantics-baseline.md | 2 +- src/FScript.Language/TypeInfer.fs | 24 +++++++------------ .../ResolvedInferenceTests.fs | 6 +++-- 5 files changed, 17 insertions(+), 19 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b96204c..b69d174 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Unified legacy and resolved inference through one semantic inference result. + - Preserved canonical ordered-map key order across `Map.ofList` and `Map.add` in LIR and Go. - Hardened canonical IR metadata validation and required generated projects to pass `go test ./...`. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 33be666..6d9ca3b 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -10,7 +10,7 @@ source -> parser/includes -> inference + resolved typed tree -> HIR -> LIR +-> Go source backend ``` -`FScript.Language` remains the parser, type-system, legacy evaluator, and Fable-compatible package. It emits a recursive semantic resolved tree in the same inference pass as its compatibility `TypedProgram`. Resolved statement, expression, and pattern nodes do not contain parser AST nodes; every resolved expression carries its final instantiated type and span plus lexical-reference kind, primitive operation, and structural/named-record conversion evidence. Compatibility inference and evaluation APIs continue to project their existing results. +`FScript.Language` remains the parser, type-system, legacy evaluator, and Fable-compatible package. It emits a recursive semantic resolved tree in the same inference pass as its compatibility `TypedProgram`. Resolved statement, expression, and pattern nodes do not contain parser AST nodes; every resolved expression carries its final instantiated type and span plus lexical-reference kind, primitive operation, and structural/named-record conversion evidence. Compatibility inference and evaluation APIs continue to project their existing results; `inferProgramWithExterns` specifically returns the `TypedProgram` from `inferResolvedProgramWithExterns` rather than running a parallel inference path. Resolved inference assigns deterministic program-local IDs to builtins, externs, types, union cases, declarations, parameters, pattern bindings, and locals in normalized source traversal. Declaration schemes and instantiated reference types normalize type variables from zero, so results do not depend on the process-global fresh-variable counter. The legacy `TypedProgram` remains a projection with unchanged public entry points. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 28a110e..c9ca0b9 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -8,7 +8,7 @@ This document records the behavior that the typed IR, LIR interpreter, and compi - Includes are resolved into one flattened program before inference. Alias-qualified declarations are rewritten by the include resolver; unresolved `SImport` nodes are rejected by inference. - Hindley-Milner inference generalizes top-level and local let bindings after applying substitutions. Recursive bindings are inferred monomorphically inside their recursive group and generalized afterward. -- The active typed program stores top-level types but retains parser expressions in bodies. Compiler lowering therefore requires a new resolved, recursively typed result emitted by the inference pass; types cannot safely be reconstructed afterward. +- The active typed program stores top-level types but retains parser expressions in bodies. Compiler lowering therefore requires a new resolved, recursively typed result emitted by the inference pass; types cannot safely be reconstructed afterward. The legacy typed-program API projects this result's `TypedProgram`, so it does not maintain a second inference path. - Resolved inference exposes parser-independent semantic statement, expression, and pattern trees whose nodes carry their final instantiated type, span, lexical reference kind, resolved primitive operation, and named/structural record-coercion evidence. It also exposes deterministic symbols for every lexical binding category and links each captured reference to its target ID. Repeated occurrences of one pattern binding reuse one ID so equality constraints cannot be lost during lowering. Type-variable numbering in resolved schemes and reference types is normalized independently of prior inference calls. - HIR reuses those resolved IDs unchanged for source symbols even when type and value declarations are interleaved; compiler-synthesized temporaries and constructor payload parameters occupy IDs above the resolved frontend range. - Top-level statements execute in source order. Every non-final expression must have type `unit`; the final expression is the program result. diff --git a/src/FScript.Language/TypeInfer.fs b/src/FScript.Language/TypeInfer.fs index 9f74b4c..4fa8a44 100644 --- a/src/FScript.Language/TypeInfer.fs +++ b/src/FScript.Language/TypeInfer.fs @@ -1603,7 +1603,7 @@ module TypeInfer = let typed, _, _ = inferProgramWithExternsRawAndLocals false externs program typed - let inferProgramWithExterns (externs: ExternalFunction list) (program: Program) : TypedProgram = + let private validateProgramBindings (externs: ExternalFunction list) (program: Program) = let reserved = BuiltinSignatures.builtinReservedNames externs @@ -1616,25 +1616,13 @@ module TypeInfer = |> Option.iter (fun name -> raise (TypeException { Message = $"Top-level binding '{name}' collides with reserved stdlib symbol"; Span = unknownSpan })) - inferProgramWithExternsRaw externs program - let inferProgramWithExternsAndLocalVariableTypes (externs: ExternalFunction list) (program: Program) : TypedProgram * LocalVariableTypeInfo list = - let reserved = BuiltinSignatures.builtinReservedNames - - externs - |> List.tryFind (fun ext -> Set.contains ext.Name reserved) - |> Option.iter (fun ext -> - raise (TypeException { Message = $"Host extern '{ext.Name}' collides with reserved stdlib symbol"; Span = unknownSpan })) - - topLevelBindingNames program - |> List.tryFind (fun name -> Set.contains name reserved) - |> Option.iter (fun name -> - raise (TypeException { Message = $"Top-level binding '{name}' collides with reserved stdlib symbol"; Span = unknownSpan })) - + validateProgramBindings externs program let typed, locals, _ = inferProgramWithExternsRawAndLocals true externs program typed, locals let inferResolvedProgramWithExterns (externs: ExternalFunction list) (program: Program) : ResolvedProgram = + validateProgramBindings externs program let statementAttributes, semanticProgram = unwrapProgramAttributes program let typed, _, expressions = inferProgramWithExternsRawAndLocals true externs semanticProgram let expressionTypes = expressions |> List.fold (fun state item -> Map.add item.Expr item.Type state) Map.empty @@ -2091,6 +2079,12 @@ module TypeInfer = let inferResolvedProgram (program: Program) : ResolvedProgram = inferResolvedProgramWithExterns [] program + /// The legacy typed-program API is a projection of the semantic resolved + /// result, so both compiler consumers and existing evaluators observe the + /// same inference pass and substitutions. + let inferProgramWithExterns (externs: ExternalFunction list) (program: Program) : TypedProgram = + (inferResolvedProgramWithExterns externs program).TypedProgram + let inferProgramWithLocalVariableTypes (program: Program) : TypedProgram * LocalVariableTypeInfo list = inferProgramWithExternsAndLocalVariableTypes [] program diff --git a/tests/FScript.Language.Tests/ResolvedInferenceTests.fs b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs index 0e6ce4e..7ff9dc9 100644 --- a/tests/FScript.Language.Tests/ResolvedInferenceTests.fs +++ b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs @@ -26,10 +26,12 @@ type ResolvedInferenceTests() = Assert.That(finalApplications, Does.Contain(TInt)) [] - member _.``Ordinary inference does not allocate resolved telemetry`` () = - let program = Helpers.parse "let identity value = value\nidentity 42" + member _.``Legacy inference preserves the resolved typed program shape`` () = + let program = Helpers.parse "let add x y = x + y\nadd 40 2" let resolved = TypeInfer.inferResolvedProgram program + let legacy = TypeInfer.inferProgram program + Assert.That(legacy.Length, Is.EqualTo(resolved.TypedProgram.Length)) Assert.That(resolved.ExpressionTypes |> List.exists (fun item -> item.Type = TInt), Is.True) [] From 14a65048697693f181ab1b48498fc94959ce110d Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 21:44:37 +0200 Subject: [PATCH 059/133] feat(compiler): lower executable pattern plans --- CHANGELOG.md | 4 + docs/architecture/typed-ir-and-backends.md | 4 +- docs/specs/compiler-semantics-baseline.md | 3 +- src/FScript.Backend.Go/GoBackend.fs | 123 +++---- src/FScript.Backend.Go/Runtime/patterns.go | 307 ++++++++++-------- src/FScript.Compiler/Hir.fs | 5 + src/FScript.Compiler/HirLowering.fs | 9 + src/FScript.Compiler/HirValidation.fs | 2 +- src/FScript.Compiler/IrArtifact.fs | 163 ++++++---- src/FScript.Compiler/Lir.fs | 252 ++++++++------ src/FScript.Interpreter.Lir/LirInterpreter.fs | 109 ++++--- src/FScript.Language/TypeInfer.fs | 10 +- .../GoBackendTests.fs | 4 +- tests/FScript.Compiler.Tests/HirTests.fs | 19 ++ .../FScript.Compiler.Tests/IrArtifactTests.fs | 9 + .../ConformanceTests.fs | 1 + 16 files changed, 637 insertions(+), 387 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b69d174..962f8b4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,10 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Lowered source patterns into backend-neutral executable LIR projection and decision plans. + +- Preserved value-first pipeline evaluation through resolved inference and compiled backends. + - Unified legacy and resolved inference through one semantic inference result. - Preserved canonical ordered-map key order across `Map.ofList` and `Map.add` in LIR and Go. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 6d9ca3b..1804176 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -22,7 +22,7 @@ Validated LIR consumers are separate leaf assemblies. `FScript.Interpreter.Lir` HIR is language-oriented and contains resolved artifact-local symbol identities, ordinary and imported type declarations, complete semantic types and schemes, source locations, structured export metadata, and explicit nodes for named/structural record conversions. HIR lowering consumes only the semantic resolved statements, expressions, and patterns; it does not traverse `Expr`, `Pattern`, `TypedStmt`, or compatibility expression telemetry. HIR directly reuses the frontend IDs for builtins, externs, types, constructors, declarations, parameters, pattern bindings, and locals; only compiler-synthesized nodes allocate IDs above the frontend range. Named record and union references remain nominal in HIR and LIR; their declaration shapes are carried separately and serialized in artifacts. Export metadata retains the exported source name and span through LIR and artifacts while the legacy Boolean remains available to compatibility consumers. Conversion evidence is produced by inference, not rediscovered from backend representations. HIR contains no Go types. -LIR is executable but not a Go AST. Lambda bodies are lifted into a deterministic program-level function table. Closure creation references a function ID and carries its canonical ordered set of captured symbol IDs, alongside resolved primitive operations, aggregate construction/projection, and structured control flow. Validation recomputes each lifted function's free-symbol set and rejects unknown function IDs or forged, missing, duplicate, or unordered captures. Pattern matching becomes an ordered decision tree with typed, serialized pattern plans before reaching a backend. Semantic types remain alongside selected runtime representations for validation and diagnostics. +LIR is executable but not a Go AST. Lambda bodies are lifted into a deterministic program-level function table. Closure creation references a function ID and carries its canonical ordered set of captured symbol IDs, alongside resolved primitive operations, aggregate construction/projection, and structured control flow. Validation recomputes each lifted function's free-symbol set and rejects unknown function IDs or forged, missing, duplicate, or unordered captures. Pattern matching becomes an ordered decision tree whose cases contain lowered projection steps, representation checks, symbol-binding plans, and deterministic map-consumption plans; no HIR/source pattern node survives in LIR. Semantic types remain alongside selected runtime representations for validation and diagnostics. The Go backend selects concrete generated structs for ordinary named records, with deterministic synthetic field identifiers and FScript-name tags. Declared unions use generated tagged structs with a field for each payload-bearing case and implement a small runtime conversion interface for generic matching and formatting. Structural records remain runtime maps, and explicit LIR conversion nodes copy fields between the representations while evaluating their source once. Runtime record projection, patterns, formatting, and equality reflect only tagged generated fields, so nominal representation does not alter FScript structural behavior. Imported record adapters remain isolated at target-call boundaries. @@ -38,7 +38,7 @@ HIR validation rejects duplicate and unresolved symbols, invalid declaration spa Lexical `let` is retained as a structured LIR node. Its symbol is visible only in the body, or in both the binding and body for recursive functions. The interpreter closes the recursive environment explicitly; Go emission declares the function slot before assigning its closure. `raise` becomes an explicit `Fail` node so consumers preserve evaluation order and attach their own stage-specific failure diagnostic. -Matches lower to a structured decision tree whose target is evaluated once. Cases remain ordered; each typed pattern plan creates artifact-local bindings, then its guard is evaluated, and only then is its body selected. A failed guard continues at the next case. Exhaustion is the explicit FScript match-failure path. The Go emitter only serializes the LIR plan and wires its result into bindings and guards; all literal, list, tuple, record, map, option, union, and type tests execute through one runtime plan evaluator rather than backend-specific source expansion. +Matches lower to a structured decision tree whose target is evaluated once. Cases remain ordered; each executable plan applies precomputed projections and checks, creates artifact-local bindings, then evaluates its guard before selecting its body. A failed guard continues at the next case. Exhaustion is the explicit FScript match-failure path. The interpreter and Go runtime execute the same LIR plan vocabulary; neither reconstructs or independently expands FScript pattern syntax. Ordered-map patterns retain ordered clause matching, repeated-binding consistency, remainder binding, and the existing rule that explicit clauses may leave unmatched entries. Type-reference patterns carry their resolved semantic type; consumers test primitive and aggregate representation shape and retain declared union identity. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index c9ca0b9..133ddf3 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -13,6 +13,7 @@ This document records the behavior that the typed IR, LIR interpreter, and compi - HIR reuses those resolved IDs unchanged for source symbols even when type and value declarations are interleaved; compiler-synthesized temporaries and constructor payload parameters occupy IDs above the resolved frontend range. - Top-level statements execute in source order. Every non-final expression must have type `unit`; the final expression is the program result. - LIR expression children execute strictly from left to right and stop at the first failure; later arguments, aggregate fields, interpolation parts, map entries, and operands are not evaluated after that failure. +- Pipeline application retains its distinct value-then-function evaluation order through resolved inference, HIR, LIR, interpretation, serialization, and Go generation; it is not rewritten into ordinary function-first application. - Compiled top-level expressions execute exactly once in source order even when interleaved with declarations. Top-level pattern bindings evaluate their scrutinee once, fail before later statements when unmatched, and expose all bound names to subsequent statements. The first aggregate compiler slice preserves tuple field order, record field names, option presence, list element order, list cons/append behavior, and recursively structural equality through HIR, serialized LIR, interpretation, and Go generation. @@ -42,7 +43,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Conditionals and match guards require Boolean values and evaluate only the selected branch. - Match clauses are tried in source order. A guard runs only after its pattern binds successfully. Failure to find a clause is a runtime error. -- Generated Go evaluates each serialized LIR pattern plan through one runtime matcher; the source emitter does not independently expand FScript pattern rules. +- HIR patterns lower into LIR projection/check/binding/map-consumption plans. LIR consumers execute that plan vocabulary and never receive source-shaped pattern nodes; generated Go evaluates each serialized plan through one runtime matcher. - Patterns cover literals, variables, wildcards, lists, tuples, records, ordered maps, options, union cases, and type-reference tests. Let-pattern failure is a runtime error. - Compiled map patterns consume entries deterministically in map order, merge repeated bindings only when structurally equal, and bind an optional tail to the remaining ordered map. Compiled type-reference patterns use resolved types rather than source text lookup. - Record updates evaluate the base once and then updates in source order. Ranges, loops, interpolation, indexing, raise, append, and cons preserve the evaluator's existing evaluation order and failures. diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index 2f1afcb..24dc86f 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -115,59 +115,62 @@ module GoBackend = | TFun _ -> $"fsruntime.AsClosure({code})" | _ -> code - let rec private patternBindings (pattern: Lir.Pattern) = - match pattern.Node with - | Lir.Binding(id, _) -> [ id ] - | Lir.ConsPattern(head, tail) -> patternBindings head @ patternBindings tail - | Lir.TuplePattern items -> List.collect patternBindings items - | Lir.RecordPattern fields -> fields |> List.collect (snd >> patternBindings) - | Lir.MapPattern(clauses, tail) -> - clauses |> List.collect (fun (key, value) -> patternBindings key @ patternBindings value) - |> (@) (tail |> Option.map patternBindings |> Option.defaultValue []) - | Lir.SomePattern item -> patternBindings item - | Lir.UnionPattern(_, _, payload) -> payload |> Option.map patternBindings |> Option.defaultValue [] - | _ -> [] - - let rec private patternBindingDetails (pattern: Lir.Pattern) = - match pattern.Node with - | Lir.Binding(id, _) -> [ id, pattern.Type ] - | Lir.ConsPattern(head, tail) -> patternBindingDetails head @ patternBindingDetails tail - | Lir.TuplePattern items -> List.collect patternBindingDetails items - | Lir.RecordPattern fields -> fields |> List.collect (snd >> patternBindingDetails) - | Lir.MapPattern(clauses, tail) -> - clauses |> List.collect (fun (key, value) -> patternBindingDetails key @ patternBindingDetails value) - |> (@) (tail |> Option.map patternBindingDetails |> Option.defaultValue []) - | Lir.SomePattern item -> patternBindingDetails item - | Lir.UnionPattern(_, _, payload) -> payload |> Option.map patternBindingDetails |> Option.defaultValue [] - | _ -> [] - - let rec private emitPatternDescriptor (pattern: Lir.Pattern) = + let rec private patternBindingDetails (plan: Lir.PatternPlan) = + let direct = List.map (fun (binding: Lir.PatternBinding) -> binding.Id, binding.Type) plan.Bindings + let nested = + plan.Maps + |> List.collect (fun (mapPlan: Lir.MapPlan) -> + [ yield! mapPlan.Clauses |> List.collect (fun (clause: Lir.MapClausePlan) -> patternBindingDetails clause.Key @ patternBindingDetails clause.Value) + yield! mapPlan.Tail |> Option.map patternBindingDetails |> Option.defaultValue [] ]) + direct @ nested + + let private patternBindings (plan: Lir.PatternPlan) = patternBindingDetails plan |> List.map fst |> List.distinct + + let private emitProjection (projection: Lir.ProjectionStep list) = let quoted value = System.Text.Json.JsonSerializer.Serialize(value) - match pattern.Node with - | Lir.Wildcard -> "fsruntime.Pattern{Kind: \"wildcard\"}" - | Lir.Binding(id, _) -> $"fsruntime.Pattern{{Kind: \"binding\", Symbol: {Lir.symbolNumber id}}}" - | Lir.LiteralPattern value -> $"fsruntime.Pattern{{Kind: \"literal\", Literal: {constant value}}}" - | Lir.EmptyList -> "fsruntime.Pattern{Kind: \"empty-list\"}" - | Lir.NonePattern -> "fsruntime.Pattern{Kind: \"none\"}" - | Lir.SomePattern item -> $"fsruntime.Pattern{{Kind: \"some\", Items: []fsruntime.Pattern{{{emitPatternDescriptor item}}}}}" - | Lir.ConsPattern(head, tail) -> $"fsruntime.Pattern{{Kind: \"cons\", Items: []fsruntime.Pattern{{{emitPatternDescriptor head}, {emitPatternDescriptor tail}}}}}" - | Lir.TuplePattern items -> - let values = items |> List.map emitPatternDescriptor |> String.concat ", " - $"fsruntime.Pattern{{Kind: \"tuple\", Items: []fsruntime.Pattern{{{values}}}}}" - | Lir.RecordPattern fields -> - let values = fields |> List.map (fun (name, item) -> $"fsruntime.FieldPattern{{Name: {quoted name}, Pattern: {emitPatternDescriptor item}}}") |> String.concat ", " - $"fsruntime.Pattern{{Kind: \"record\", Fields: []fsruntime.FieldPattern{{{values}}}}}" - | Lir.MapPattern(clauses, tail) -> - let values = clauses |> List.map (fun (key, value) -> $"fsruntime.MapPatternClause{{Key: {emitPatternDescriptor key}, Value: {emitPatternDescriptor value}}}") |> String.concat ", " - let tailValue = tail |> Option.map (fun item -> $"fsruntime.PatternPtr({emitPatternDescriptor item})") |> Option.defaultValue "nil" - $"fsruntime.Pattern{{Kind: \"map\", Clauses: []fsruntime.MapPatternClause{{{values}}}, Tail: {tailValue}}}" - | Lir.UnionPattern(typeName, caseName, payload) -> - let payloadValue = payload |> Option.map (fun item -> $"[]fsruntime.Pattern{{{emitPatternDescriptor item}}}") |> Option.defaultValue "nil" - $"fsruntime.Pattern{{Kind: \"union\", TypeName: {quoted typeName}, CaseName: {quoted caseName}, Items: {payloadValue}}}" - | Lir.TypePattern patternType -> $"fsruntime.Pattern{{Kind: \"type\", TypeName: {quoted (Types.typeToString patternType)}}}" - - let private emitPatternBindings bindingsName (pattern: Lir.Pattern) = - patternBindingDetails pattern + let steps = + projection + |> List.map (function + | Lir.ListHead -> "fsruntime.ProjectionStep{Kind: \"list-head\"}" + | Lir.ListTail -> "fsruntime.ProjectionStep{Kind: \"list-tail\"}" + | Lir.TupleItem index -> $"fsruntime.ProjectionStep{{Kind: \"tuple-item\", Index: {index}}}" + | Lir.RecordField name -> $"fsruntime.ProjectionStep{{Kind: \"record-field\", Name: {quoted name}}}" + | Lir.OptionValue -> "fsruntime.ProjectionStep{Kind: \"option-value\"}" + | Lir.UnionPayload -> "fsruntime.ProjectionStep{Kind: \"union-payload\"}") + |> String.concat ", " + $"[]fsruntime.ProjectionStep{{{steps}}}" + + let private emitPatternCheck check = + let quoted value = System.Text.Json.JsonSerializer.Serialize(value) + match check with + | Lir.LiteralCheck(projection, value) -> $"fsruntime.PatternCheck{{Kind: \"literal\", Projection: {emitProjection projection}, Literal: {constant value}}}" + | Lir.EmptyListCheck projection -> $"fsruntime.PatternCheck{{Kind: \"empty-list\", Projection: {emitProjection projection}}}" + | Lir.NonEmptyListCheck projection -> $"fsruntime.PatternCheck{{Kind: \"non-empty-list\", Projection: {emitProjection projection}}}" + | Lir.TupleArityCheck(projection, arity) -> $"fsruntime.PatternCheck{{Kind: \"tuple-arity\", Projection: {emitProjection projection}, Arity: {arity}}}" + | Lir.RecordFieldCheck(projection, name) -> $"fsruntime.PatternCheck{{Kind: \"record-field\", Projection: {emitProjection projection}, Name: {quoted name}}}" + | Lir.SomeCheck projection -> $"fsruntime.PatternCheck{{Kind: \"some\", Projection: {emitProjection projection}}}" + | Lir.NoneCheck projection -> $"fsruntime.PatternCheck{{Kind: \"none\", Projection: {emitProjection projection}}}" + | Lir.UnionCaseCheck(projection, typeName, caseName, hasPayload) -> + $"fsruntime.PatternCheck{{Kind: \"union-case\", Projection: {emitProjection projection}, TypeName: {quoted typeName}, CaseName: {quoted caseName}, HasPayload: {hasPayload.ToString().ToLowerInvariant()}}}" + | Lir.TypeCheck(projection, patternType) -> $"fsruntime.PatternCheck{{Kind: \"type\", Projection: {emitProjection projection}, TypeName: {quoted (Types.typeToString patternType)}}}" + + let rec private emitPatternPlan (plan: Lir.PatternPlan) = + let checks = plan.Checks |> List.map emitPatternCheck |> String.concat ", " + let bindings = + plan.Bindings + |> List.map (fun binding -> $"fsruntime.PatternBindingPlan{{Symbol: {Lir.symbolNumber binding.Id}, Projection: {emitProjection binding.Projection}}}") + |> String.concat ", " + let maps = + plan.Maps + |> List.map (fun mapPlan -> + let clauses = mapPlan.Clauses |> List.map (fun clause -> $"fsruntime.MapPatternClausePlan{{Key: {emitPatternPlan clause.Key}, Value: {emitPatternPlan clause.Value}}}") |> String.concat ", " + let tail = mapPlan.Tail |> Option.map (fun item -> $"fsruntime.PatternPlanPtr({emitPatternPlan item})") |> Option.defaultValue "nil" + $"fsruntime.MapPatternPlan{{Projection: {emitProjection mapPlan.Projection}, Clauses: []fsruntime.MapPatternClausePlan{{{clauses}}}, Tail: {tail}, RequireEmptyRemainder: {mapPlan.RequireEmptyRemainder.ToString().ToLowerInvariant()}}}") + |> String.concat ", " + $"fsruntime.PatternPlan{{Checks: []fsruntime.PatternCheck{{{checks}}}, Bindings: []fsruntime.PatternBindingPlan{{{bindings}}}, Maps: []fsruntime.MapPatternPlan{{{maps}}}}}" + + let private emitPatternBindings bindingsName (plan: Lir.PatternPlan) = + patternBindingDetails plan |> List.distinctBy fst |> List.map (fun (id, bindingType) -> let name = identifier "match" id @@ -243,6 +246,12 @@ module GoBackend = match emitExpression environment fn, emitExpression environment argument with | Ok fnCode, Ok argumentCode -> Ok(cast expression.Type $"fsruntime.Call({fnCode}, {argumentCode})") | Error error, _ | _, Error error -> Error error + | Lir.PipeClosure(argument, fn) -> + match emitExpression environment argument, emitExpression environment fn with + | Ok argumentCode, Ok fnCode -> + let callCode = cast expression.Type "fsruntime.Call(pipelineFunction, pipelineArgument)" + Ok $"func() {goType expression.Type} {{\n\tpipelineArgument := {indent argumentCode}\n\tpipelineFunction := {indent fnCode}\n\treturn {callCode}\n}}()" + | Error error, _ | _, Error error -> Error error | Lir.Branch(condition, whenTrue, whenFalse) -> match emitExpression environment condition, emitExpression environment whenTrue, emitExpression environment whenFalse with | Ok c, Ok t, Ok f -> Ok $"func() {goType expression.Type} {{\n\tif {c} {{\n\t\treturn {indent (indent t)}\n\t}}\n\treturn {indent f}\n}}()" @@ -332,15 +341,15 @@ module GoBackend = let mutable failure = None let clauses = StringBuilder() for caseIndex, case in cases |> List.indexed do - let bindings = patternBindings case.Pattern + let bindings = patternBindings case.Plan let caseEnvironment = bindings |> List.fold (fun env id -> Map.add id (identifier "match" id) env) environment match case.Guard |> Option.map (emitExpression caseEnvironment) |> Option.defaultValue (Ok "true"), emitExpression caseEnvironment case.Body with | Ok guardCode, Ok bodyCode -> - let descriptor = emitPatternDescriptor case.Pattern + let descriptor = emitPatternPlan case.Plan let bindingsName = $"patternBindings{caseIndex}" let matchedName = $"patternMatched{caseIndex}" - let assignments = emitPatternBindings bindingsName case.Pattern - clauses.Append($"\t{bindingsName}, {matchedName} := fsruntime.MatchPattern(matchValue, {descriptor})\n\t_ = {bindingsName}\n\tif {matchedName} {{\n") |> ignore + let assignments = emitPatternBindings bindingsName case.Plan + clauses.Append($"\t{bindingsName}, {matchedName} := fsruntime.MatchPatternPlan(matchValue, {descriptor})\n\t_ = {bindingsName}\n\tif {matchedName} {{\n") |> ignore assignments |> List.iter (fun assignment -> clauses.Append("\t\t").Append(assignment.Replace("; ", "\n\t\t")).Append("\n") |> ignore) clauses.Append($"\t\tif {guardCode} {{\n\t\t\treturn {indent (indent (indent bodyCode))}\n\t\t}}\n\t}}\n") |> ignore | Error error, _ | _, Error error -> failure <- Some error @@ -662,7 +671,7 @@ func Environment() map[string]any { | Lir.Load id -> Set.singleton id | Lir.MakeClosure(_, captures) -> Set.ofList captures | Lir.Fail body | Lir.FieldGet(body, _) | Lir.SomeValue body -> references body - | Lir.CallClosure(left, right) | Lir.Append(left, right) | Lir.Cons(left, right) | Lir.Range(left, right) | Lir.IndexGet(left, right) -> Set.union (references left) (references right) + | Lir.CallClosure(left, right) | Lir.PipeClosure(left, right) | Lir.Append(left, right) | Lir.Cons(left, right) | Lir.Range(left, right) | Lir.IndexGet(left, right) -> Set.union (references left) (references right) | Lir.Branch(condition, whenTrue, whenFalse) -> many [ condition; whenTrue; whenFalse ] | Lir.Operation(_, values) | Lir.Tuple values | Lir.List values -> many values | Lir.Record fields -> fields |> List.map (snd >> references) |> Set.unionMany diff --git a/src/FScript.Backend.Go/Runtime/patterns.go b/src/FScript.Backend.Go/Runtime/patterns.go index 8612c57..3672998 100644 --- a/src/FScript.Backend.Go/Runtime/patterns.go +++ b/src/FScript.Backend.Go/Runtime/patterns.go @@ -2,182 +2,231 @@ package fsruntime import "strings" -type Pattern struct { - Kind string - Symbol int - Literal any - Items []Pattern - Fields []FieldPattern - Clauses []MapPatternClause - Tail *Pattern - TypeName string - CaseName string +type ProjectionStep struct { + Kind string + Index int + Name string } -type FieldPattern struct { - Name string - Pattern Pattern + +type PatternCheck struct { + Kind string + Projection []ProjectionStep + Literal any + Arity int + Name string + TypeName string + CaseName string + HasPayload bool +} + +type PatternBindingPlan struct { + Symbol int + Projection []ProjectionStep +} + +type PatternPlan struct { + Checks []PatternCheck + Bindings []PatternBindingPlan + Maps []MapPatternPlan +} + +type MapPatternPlan struct { + Projection []ProjectionStep + Clauses []MapPatternClausePlan + Tail *PatternPlan + RequireEmptyRemainder bool } -type MapPatternClause struct { - Key Pattern - Value Pattern + +type MapPatternClausePlan struct { + Key PatternPlan + Value PatternPlan } -func PatternPtr(value Pattern) *Pattern { return &value } +func PatternPlanPtr(value PatternPlan) *PatternPlan { return &value } -func mergePatternBindings(left map[int]any, right map[int]any) (map[int]any, bool) { +func copyPatternBindings(values map[int]any) map[int]any { result := map[int]any{} - for key, value := range left { + for key, value := range values { result[key] = value } - for key, value := range right { - if existing, found := result[key]; found && !Equal(existing, value) { + return result +} + +func addPatternBinding(values map[int]any, symbol int, value any) (map[int]any, bool) { + if existing, found := values[symbol]; found && !Equal(existing, value) { + return nil, false + } + result := copyPatternBindings(values) + result[symbol] = value + return result, true +} + +func projectPatternValue(value any, projection []ProjectionStep) (any, bool) { + current := value + for _, step := range projection { + switch step.Kind { + case "list-head": + values, ok := current.([]any) + if !ok || len(values) == 0 { + return nil, false + } + current = values[0] + case "list-tail": + values, ok := current.([]any) + if !ok || len(values) == 0 { + return nil, false + } + current = values[1:] + case "tuple-item": + values, ok := current.(Tuple) + if !ok || step.Index < 0 || step.Index >= len(values) { + return nil, false + } + current = values[step.Index] + case "record-field": + values, ok := TryRecord(current) + if !ok { + return nil, false + } + item, found := values[step.Name] + if !found { + return nil, false + } + current = item + case "option-value": + option, ok := current.(Option) + if !ok || !option.HasValue { + return nil, false + } + current = option.Value + case "union-payload": + union, ok := TryUnion(current) + if !ok || !union.HasValue { + return nil, false + } + current = union.Value + default: return nil, false } - result[key] = value } - return result, true + return current, true } -func matchRuntimePattern(value any, pattern Pattern) (map[int]any, bool) { - switch pattern.Kind { - case "wildcard": - return map[int]any{}, true - case "binding": - return map[int]any{pattern.Symbol: value}, true +func checkPatternPlan(root any, check PatternCheck) bool { + value, ok := projectPatternValue(root, check.Projection) + if !ok { + return false + } + switch check.Kind { case "literal": - return map[int]any{}, Equal(value, pattern.Literal) + return Equal(value, check.Literal) case "empty-list": values, ok := value.([]any) - return map[int]any{}, ok && len(values) == 0 - case "none": - option, ok := value.(Option) - return map[int]any{}, ok && !option.HasValue - case "some": - option, ok := value.(Option) - if !ok || !option.HasValue { - return nil, false - } - return matchRuntimePattern(option.Value, pattern.Items[0]) - case "cons": + return ok && len(values) == 0 + case "non-empty-list": values, ok := value.([]any) - if !ok || len(values) == 0 { - return nil, false + return ok && len(values) > 0 + case "tuple-arity": + values, ok := value.(Tuple) + return ok && len(values) == check.Arity + case "record-field": + values, ok := TryRecord(value) + if !ok { + return false } - left, matched := matchRuntimePattern(values[0], pattern.Items[0]) - if !matched { + _, found := values[check.Name] + return found + case "some": + option, ok := value.(Option) + return ok && option.HasValue + case "none": + option, ok := value.(Option) + return ok && !option.HasValue + case "union-case": + item, ok := TryUnion(value) + return ok && item.TypeName == check.TypeName && item.CaseName == check.CaseName && item.HasValue == check.HasPayload + case "type": + return TypeMatches(value, check.TypeName) + default: + return false + } +} + +func matchPatternPlan(value any, plan PatternPlan, initial map[int]any) (map[int]any, bool) { + for _, check := range plan.Checks { + if !checkPatternPlan(value, check) { return nil, false } - right, matched := matchRuntimePattern(values[1:], pattern.Items[1]) - if !matched { + } + bindings := copyPatternBindings(initial) + for _, binding := range plan.Bindings { + item, ok := projectPatternValue(value, binding.Projection) + if !ok { return nil, false } - return mergePatternBindings(left, right) - case "tuple": - values, ok := value.(Tuple) - if !ok || len(values) != len(pattern.Items) { + bindings, ok = addPatternBinding(bindings, binding.Symbol, item) + if !ok { return nil, false } - bindings := map[int]any{} - for index, item := range pattern.Items { - next, matched := matchRuntimePattern(values[index], item) - if !matched { - return nil, false - } - bindings, matched = mergePatternBindings(bindings, next) - if !matched { - return nil, false - } - } - return bindings, true - case "record": - values, ok := TryRecord(value) + } + for _, mapPlan := range plan.Maps { + projected, ok := projectPatternValue(value, mapPlan.Projection) if !ok { return nil, false } - bindings := map[int]any{} - for _, field := range pattern.Fields { - item, found := values[field.Name] - if !found { - return nil, false - } - next, matched := matchRuntimePattern(item, field.Pattern) - if !matched { - return nil, false - } - bindings, matched = mergePatternBindings(bindings, next) - if !matched { - return nil, false - } - } - return bindings, true - case "union": - item, ok := TryUnion(value) - if !ok || item.TypeName != pattern.TypeName || item.CaseName != pattern.CaseName || (len(pattern.Items) == 0) != !item.HasValue { + item, ok := projected.(OrderedMap) + if !ok { return nil, false } - if len(pattern.Items) == 0 { - return map[int]any{}, true - } - return matchRuntimePattern(item.Value, pattern.Items[0]) - case "type": - return map[int]any{}, TypeMatches(value, pattern.TypeName) - case "map": - item, ok := value.(OrderedMap) + bindings, ok = matchMapPatternPlan(item.Entries, mapPlan, bindings) if !ok { return nil, false } - remaining := append([]MapEntry{}, item.Entries...) - bindings := map[int]any{} - for _, clause := range pattern.Clauses { - found := false - for index, entry := range remaining { - keyBindings, keyMatched := matchRuntimePattern(entry.Key, clause.Key) - if !keyMatched { - continue - } - withKey, merged := mergePatternBindings(bindings, keyBindings) - if !merged { - continue - } - valueBindings, valueMatched := matchRuntimePattern(entry.Value, clause.Value) - if !valueMatched { - continue - } - withValue, merged := mergePatternBindings(withKey, valueBindings) - if !merged { - continue - } - bindings = withValue - remaining = append(remaining[:index], remaining[index+1:]...) - found = true - break + } + return bindings, true +} + +func matchMapPatternPlan(entries []MapEntry, plan MapPatternPlan, initial map[int]any) (map[int]any, bool) { + var matchClauses func([]MapEntry, []MapPatternClausePlan, map[int]any) (map[int]any, bool) + matchClauses = func(remaining []MapEntry, clauses []MapPatternClausePlan, bindings map[int]any) (map[int]any, bool) { + if len(clauses) == 0 { + if plan.Tail != nil { + return matchPatternPlan(OrderedMap{Entries: remaining}, *plan.Tail, bindings) } - if !found { + if plan.RequireEmptyRemainder && len(remaining) != 0 { return nil, false } + return bindings, true } - if pattern.Tail != nil { - tailBindings, matched := matchRuntimePattern(OrderedMap{Entries: remaining}, *pattern.Tail) + clause := clauses[0] + for index, entry := range remaining { + withKey, matched := matchPatternPlan(entry.Key, clause.Key, bindings) if !matched { - return nil, false + continue + } + withValue, matched := matchPatternPlan(entry.Value, clause.Value, withKey) + if !matched { + continue + } + next := append([]MapEntry{}, remaining[:index]...) + next = append(next, remaining[index+1:]...) + if result, matched := matchClauses(next, clauses[1:], withValue); matched { + return result, true } - return mergePatternBindings(bindings, tailBindings) - } - if len(pattern.Clauses) > 0 || len(remaining) == 0 { - return bindings, true } return nil, false - default: - return nil, false } + return matchClauses(append([]MapEntry{}, entries...), plan.Clauses, initial) } -func MatchPattern(value any, pattern Pattern) (map[int]any, bool) { - return matchRuntimePattern(value, pattern) +func MatchPatternPlan(value any, plan PatternPlan) (map[int]any, bool) { + return matchPatternPlan(value, plan, map[int]any{}) } + func PatternBinding(bindings map[int]any, symbol int) any { return bindings[symbol] } + func TypeMatches(value any, typeName string) bool { switch typeName { case "unit": diff --git a/src/FScript.Compiler/Hir.fs b/src/FScript.Compiler/Hir.fs index b056b87..60f0e6f 100644 --- a/src/FScript.Compiler/Hir.fs +++ b/src/FScript.Compiler/Hir.fs @@ -40,6 +40,7 @@ module Hir = | Reference of SymbolId | Lambda of SymbolId * string * Type * Expression | Apply of Expression * Expression + | Pipe of Expression * Expression | If of Expression * Expression * Expression | Primitive of PrimitiveOperation * Expression list | Tuple of Expression list @@ -173,6 +174,10 @@ module Hir = lines.Add(typed "apply") expression (indent + 2) fn expression (indent + 2) argument + | Pipe(value, fn) -> + lines.Add(typed "pipe") + expression (indent + 2) value + expression (indent + 2) fn | If(condition, whenTrue, whenFalse) -> lines.Add(typed "if") expression (indent + 2) condition diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index 1a2053a..4ba114a 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -126,6 +126,15 @@ module HirLowering = ({ Type = targetType; Span = loweredArgument.Span; Node = Hir.RecordConvert(loweredArgument, targetType) }: Hir.Expression)) |> Option.defaultValue loweredArgument make (Hir.Apply(lower fn, argumentWithEvidence)) + | TypeInfer.ResolvedPipe(argument, fn) -> + let loweredArgument = lower argument + let argumentWithEvidence = + expression.RecordCoercion + |> Option.map (fun evidence -> + let targetType = representationType state evidence.TargetType + ({ Type = targetType; Span = loweredArgument.Span; Node = Hir.RecordConvert(loweredArgument, targetType) }: Hir.Expression)) + |> Option.defaultValue loweredArgument + make (Hir.Pipe(argumentWithEvidence, lower fn)) | TypeInfer.ResolvedIf(condition, whenTrue, whenFalse) -> make (Hir.If(lower condition, lower whenTrue, lower whenFalse)) | TypeInfer.ResolvedPrimitive(operation, arguments) -> make (Hir.Primitive(resolvedOperation operation, List.map lower arguments)) | TypeInfer.ResolvedTuple items -> make (Hir.Tuple(List.map lower items)) diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index 0ba4e78..c9555cb 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -63,7 +63,7 @@ module HirValidation = | Hir.Constant _ -> Set.empty | Hir.Reference id -> Set.singleton id | Hir.Lambda(parameter, _, _, body) -> references body |> Set.remove parameter - | Hir.Apply(fn, argument) -> Set.union (references fn) (references argument) + | Hir.Apply(fn, argument) | Hir.Pipe(argument, fn) -> Set.union (references fn) (references argument) | Hir.If(condition, whenTrue, whenFalse) -> Set.unionMany [ references condition; references whenTrue; references whenFalse ] | Hir.Primitive(_, arguments) -> arguments |> List.map references |> Set.unionMany diff --git a/src/FScript.Compiler/IrArtifact.fs b/src/FScript.Compiler/IrArtifact.fs index 60513b4..5bc1498 100644 --- a/src/FScript.Compiler/IrArtifact.fs +++ b/src/FScript.Compiler/IrArtifact.fs @@ -105,39 +105,58 @@ module IrArtifact = | Lir.Unit -> () | Lir.Int value -> writer.WriteNumber("intValue", value) | Lir.Float value -> writer.WriteNumber("floatValue", value) | Lir.Bool value -> writer.WriteBoolean("boolValue", value) | Lir.String value -> writer.WriteString("stringValue", value) - let rec private writePattern (writer: Utf8JsonWriter) (value: Lir.Pattern) = - writer.WriteStartObject(); writer.WritePropertyName("type"); writeType writer value.Type - writer.WritePropertyName("span"); writeSpan writer value.Span - match value.Node with - | Lir.Wildcard -> writer.WriteString("kind", "wildcard") - | Lir.Binding(id, name) -> writer.WriteString("kind", "binding"); writer.WriteNumber("symbol", Lir.symbolNumber id); writer.WriteString("name", name) - | Lir.LiteralPattern constant -> writer.WriteString("kind", "literal"); writeConstantValue writer constant - | Lir.EmptyList -> writer.WriteString("kind", "empty-list") - | Lir.NonePattern -> writer.WriteString("kind", "none") - | Lir.SomePattern item -> writer.WriteString("kind", "some"); writer.WritePropertyName("item"); writePattern writer item - | Lir.ConsPattern(head, tail) -> - writer.WriteString("kind", "cons"); writer.WritePropertyName("head"); writePattern writer head - writer.WritePropertyName("tail"); writePattern writer tail - | Lir.TuplePattern items -> - writer.WriteString("kind", "tuple"); writer.WriteStartArray("items"); items |> List.iter (writePattern writer); writer.WriteEndArray() - | Lir.RecordPattern fields -> - writer.WriteString("kind", "record"); writer.WriteStartArray("fields") - for name, item in fields do - writer.WriteStartObject(); writer.WriteString("name", name); writer.WritePropertyName("pattern"); writePattern writer item; writer.WriteEndObject() - writer.WriteEndArray() - | Lir.MapPattern(clauses, tail) -> - writer.WriteString("kind", "map"); writer.WriteStartArray("clauses") - for key, item in clauses do - writer.WriteStartObject(); writer.WritePropertyName("key"); writePattern writer key - writer.WritePropertyName("value"); writePattern writer item; writer.WriteEndObject() - writer.WriteEndArray() - match tail with Some item -> writer.WritePropertyName("tail"); writePattern writer item | None -> writer.WriteNull("tail") - | Lir.UnionPattern(typeName, caseName, payload) -> - writer.WriteString("kind", "union"); writer.WriteString("typeName", typeName); writer.WriteString("caseName", caseName) - match payload with Some item -> writer.WritePropertyName("payload"); writePattern writer item | None -> writer.WriteNull("payload") - | Lir.TypePattern patternType -> writer.WriteString("kind", "type"); writer.WritePropertyName("patternType"); writeType writer patternType + let private writeProjection (writer: Utf8JsonWriter) (name: string) (steps: Lir.ProjectionStep list) = + writer.WriteStartArray(name) + for step in steps do + writer.WriteStartObject() + match step with + | Lir.ListHead -> writer.WriteString("kind", "list-head") + | Lir.ListTail -> writer.WriteString("kind", "list-tail") + | Lir.TupleItem index -> writer.WriteString("kind", "tuple-item"); writer.WriteNumber("index", index) + | Lir.RecordField name -> writer.WriteString("kind", "record-field"); writer.WriteString("name", name) + | Lir.OptionValue -> writer.WriteString("kind", "option-value") + | Lir.UnionPayload -> writer.WriteString("kind", "union-payload") + writer.WriteEndObject() + writer.WriteEndArray() + + let private writeCheck (writer: Utf8JsonWriter) check = + writer.WriteStartObject() + match check with + | Lir.LiteralCheck(projection, value) -> writer.WriteString("kind", "literal"); writeProjection writer "projection" projection; writeConstantValue writer value + | Lir.EmptyListCheck projection -> writer.WriteString("kind", "empty-list"); writeProjection writer "projection" projection + | Lir.NonEmptyListCheck projection -> writer.WriteString("kind", "non-empty-list"); writeProjection writer "projection" projection + | Lir.TupleArityCheck(projection, arity) -> writer.WriteString("kind", "tuple-arity"); writeProjection writer "projection" projection; writer.WriteNumber("arity", arity) + | Lir.RecordFieldCheck(projection, name) -> writer.WriteString("kind", "record-field"); writeProjection writer "projection" projection; writer.WriteString("name", name) + | Lir.SomeCheck projection -> writer.WriteString("kind", "some"); writeProjection writer "projection" projection + | Lir.NoneCheck projection -> writer.WriteString("kind", "none"); writeProjection writer "projection" projection + | Lir.UnionCaseCheck(projection, typeName, caseName, hasPayload) -> + writer.WriteString("kind", "union-case"); writeProjection writer "projection" projection + writer.WriteString("typeName", typeName); writer.WriteString("caseName", caseName); writer.WriteBoolean("hasPayload", hasPayload) + | Lir.TypeCheck(projection, patternType) -> + writer.WriteString("kind", "type"); writeProjection writer "projection" projection + writer.WritePropertyName("patternType"); writeType writer patternType writer.WriteEndObject() + let rec private writePlan (writer: Utf8JsonWriter) (plan: Lir.PatternPlan) = + writer.WriteStartObject(); writer.WritePropertyName("span"); writeSpan writer plan.Span + writer.WriteStartArray("checks"); plan.Checks |> List.iter (writeCheck writer); writer.WriteEndArray() + writer.WriteStartArray("bindings") + for binding in plan.Bindings do + writer.WriteStartObject(); writer.WriteNumber("symbol", Lir.symbolNumber binding.Id); writer.WriteString("name", binding.Name) + writer.WritePropertyName("type"); writeType writer binding.Type; writeProjection writer "projection" binding.Projection + writer.WriteEndObject() + writer.WriteEndArray(); writer.WriteStartArray("maps") + for mapPlan in plan.Maps do + writer.WriteStartObject(); writeProjection writer "projection" mapPlan.Projection + writer.WriteBoolean("requireEmptyRemainder", mapPlan.RequireEmptyRemainder); writer.WriteStartArray("clauses") + for clause in mapPlan.Clauses do + writer.WriteStartObject(); writer.WritePropertyName("key"); writePlan writer clause.Key + writer.WritePropertyName("value"); writePlan writer clause.Value; writer.WriteEndObject() + writer.WriteEndArray() + match mapPlan.Tail with Some tail -> writer.WritePropertyName("tail"); writePlan writer tail | None -> writer.WriteNull("tail") + writer.WriteEndObject() + writer.WriteEndArray(); writer.WriteEndObject() + let rec private writeExpression (writer: Utf8JsonWriter) (value: Lir.Expression) = writer.WriteStartObject(); writer.WritePropertyName("type"); writeType writer value.Type writer.WritePropertyName("span"); writeSpan writer value.Span @@ -151,6 +170,9 @@ module IrArtifact = | Lir.CallClosure(fn, argument) -> writer.WriteString("kind", "call"); writer.WritePropertyName("function"); writeExpression writer fn writer.WritePropertyName("argument"); writeExpression writer argument + | Lir.PipeClosure(argument, fn) -> + writer.WriteString("kind", "pipe-call"); writer.WritePropertyName("argument"); writeExpression writer argument + writer.WritePropertyName("function"); writeExpression writer fn | Lir.Branch(condition, whenTrue, whenFalse) -> writer.WriteString("kind", "branch"); writer.WritePropertyName("condition"); writeExpression writer condition writer.WritePropertyName("whenTrue"); writeExpression writer whenTrue; writer.WritePropertyName("whenFalse"); writeExpression writer whenFalse @@ -194,7 +216,7 @@ module IrArtifact = writer.WriteString("kind", "decision-tree"); writer.WritePropertyName("target"); writeExpression writer target writer.WriteStartArray("cases") for case in cases do - writer.WriteStartObject(); writer.WritePropertyName("pattern"); writePattern writer case.Pattern + writer.WriteStartObject(); writer.WritePropertyName("plan"); writePlan writer case.Plan match case.Guard with Some guard -> writer.WritePropertyName("guard"); writeExpression writer guard | None -> writer.WriteNull("guard") writer.WritePropertyName("body"); writeExpression writer case.Body; writer.WritePropertyName("span"); writeSpan writer case.Span writer.WriteEndObject() @@ -347,28 +369,58 @@ module IrArtifact = | "string" -> Lir.String(stringProperty "stringValue" element) | value -> invalidOp $"Unknown constant '{value}'." - let rec private readPattern (element: JsonElement) : Lir.Pattern = - let node = - match stringProperty "kind" element with - | "wildcard" -> Lir.Wildcard - | "binding" -> Lir.Binding(Lir.SymbolId((property "symbol" element).GetInt32()), stringProperty "name" element) - | "literal" -> Lir.LiteralPattern(readConstant element) - | "empty-list" -> Lir.EmptyList - | "none" -> Lir.NonePattern - | "some" -> Lir.SomePattern(readPattern (property "item" element)) - | "cons" -> Lir.ConsPattern(readPattern (property "head" element), readPattern (property "tail" element)) - | "tuple" -> Lir.TuplePattern((property "items" element).EnumerateArray() |> Seq.map readPattern |> Seq.toList) - | "record" -> Lir.RecordPattern((property "fields" element).EnumerateArray() |> Seq.map (fun field -> stringProperty "name" field, readPattern (property "pattern" field)) |> Seq.toList) - | "map" -> - let clauses = (property "clauses" element).EnumerateArray() |> Seq.map (fun clause -> readPattern (property "key" clause), readPattern (property "value" clause)) |> Seq.toList - let tail = property "tail" element - Lir.MapPattern(clauses, if tail.ValueKind = JsonValueKind.Null then None else Some(readPattern tail)) - | "union" -> - let payload = property "payload" element - Lir.UnionPattern(stringProperty "typeName" element, stringProperty "caseName" element, if payload.ValueKind = JsonValueKind.Null then None else Some(readPattern payload)) - | "type" -> Lir.TypePattern(readType (property "patternType" element)) - | kind -> invalidOp $"Unknown LIR pattern kind '{kind}'." - { Type = readType (property "type" element); Span = readSpan (property "span" element); Node = node } + let private readProjection (element: JsonElement) name = + (property name element).EnumerateArray() + |> Seq.map (fun step -> + match stringProperty "kind" step with + | "list-head" -> Lir.ListHead + | "list-tail" -> Lir.ListTail + | "tuple-item" -> Lir.TupleItem((property "index" step).GetInt32()) + | "record-field" -> Lir.RecordField(stringProperty "name" step) + | "option-value" -> Lir.OptionValue + | "union-payload" -> Lir.UnionPayload + | kind -> invalidOp $"Unknown LIR projection step '{kind}'.") + |> Seq.toList + + let private readCheck (element: JsonElement) = + let projection = readProjection element "projection" + match stringProperty "kind" element with + | "literal" -> Lir.LiteralCheck(projection, readConstant element) + | "empty-list" -> Lir.EmptyListCheck projection + | "non-empty-list" -> Lir.NonEmptyListCheck projection + | "tuple-arity" -> Lir.TupleArityCheck(projection, (property "arity" element).GetInt32()) + | "record-field" -> Lir.RecordFieldCheck(projection, stringProperty "name" element) + | "some" -> Lir.SomeCheck projection + | "none" -> Lir.NoneCheck projection + | "union-case" -> Lir.UnionCaseCheck(projection, stringProperty "typeName" element, stringProperty "caseName" element, (property "hasPayload" element).GetBoolean()) + | "type" -> Lir.TypeCheck(projection, readType (property "patternType" element)) + | kind -> invalidOp $"Unknown LIR pattern-plan check '{kind}'." + + let rec private readPlan (element: JsonElement) : Lir.PatternPlan = + let bindings = + (property "bindings" element).EnumerateArray() + |> Seq.map (fun binding -> + ({ Id = Lir.SymbolId((property "symbol" binding).GetInt32()) + Name = stringProperty "name" binding + Type = readType (property "type" binding) + Projection = readProjection binding "projection" }: Lir.PatternBinding)) + |> Seq.toList + let maps = + (property "maps" element).EnumerateArray() + |> Seq.map (fun mapPlan -> + let tail = property "tail" mapPlan + ({ Projection = readProjection mapPlan "projection" + Clauses = + (property "clauses" mapPlan).EnumerateArray() + |> Seq.map (fun clause -> ({ Key = readPlan (property "key" clause); Value = readPlan (property "value" clause) }: Lir.MapClausePlan)) + |> Seq.toList + Tail = if tail.ValueKind = JsonValueKind.Null then None else Some(readPlan tail) + RequireEmptyRemainder = (property "requireEmptyRemainder" mapPlan).GetBoolean() }: Lir.MapPlan)) + |> Seq.toList + { Span = readSpan (property "span" element) + Checks = (property "checks" element).EnumerateArray() |> Seq.map readCheck |> Seq.toList + Bindings = bindings + Maps = maps } let rec private readExpression (element: JsonElement) : Lir.Expression = let expressionType = readType (property "type" element) @@ -382,6 +434,7 @@ module IrArtifact = Lir.FunctionId((property "function" element).GetInt32()), (property "captures" element).EnumerateArray() |> Seq.map (fun item -> Lir.SymbolId(item.GetInt32())) |> Seq.toList) | "call" -> Lir.CallClosure(readExpression (property "function" element), readExpression (property "argument" element)) + | "pipe-call" -> Lir.PipeClosure(readExpression (property "argument" element), readExpression (property "function" element)) | "branch" -> Lir.Branch(readExpression (property "condition" element), readExpression (property "whenTrue" element), readExpression (property "whenFalse" element)) | "operation" -> Lir.Operation(readOperation (stringProperty "operation" element), (property "arguments" element).EnumerateArray() |> Seq.map readExpression |> Seq.toList) | "tuple" -> Lir.Tuple((property "items" element).EnumerateArray() |> Seq.map readExpression |> Seq.toList) @@ -404,7 +457,7 @@ module IrArtifact = | "decision-tree" -> let cases = (property "cases" element).EnumerateArray() |> Seq.map (fun item -> let guard = property "guard" item - ({ Pattern = readPattern (property "pattern" item) + ({ Plan = readPlan (property "plan" item) Guard = if guard.ValueKind = JsonValueKind.Null then None else Some(readExpression guard) Body = readExpression (property "body" item) Span = readSpan (property "span" item) }: Lir.DecisionCase)) |> Seq.toList diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index 99a71d7..aee9a01 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -29,6 +29,7 @@ module Lir = | Load of SymbolId | MakeClosure of FunctionId * SymbolId list | CallClosure of Expression * Expression + | PipeClosure of Expression * Expression | Branch of Expression * Expression * Expression | Operation of Operation * Expression list | Tuple of Expression list @@ -65,27 +66,49 @@ module Lir = | TextPart of string | ValuePart of Expression - and Pattern = - { Type: Type - Span: Span - Node: PatternNode } - - and PatternNode = - | Wildcard - | Binding of SymbolId * string - | LiteralPattern of Constant - | EmptyList - | ConsPattern of Pattern * Pattern - | TuplePattern of Pattern list - | RecordPattern of (string * Pattern) list - | MapPattern of (Pattern * Pattern) list * Pattern option - | SomePattern of Pattern - | NonePattern - | UnionPattern of string * string * Pattern option - | TypePattern of Type + and ProjectionStep = + | ListHead + | ListTail + | TupleItem of int + | RecordField of string + | OptionValue + | UnionPayload + + and PatternCheck = + | LiteralCheck of ProjectionStep list * Constant + | EmptyListCheck of ProjectionStep list + | NonEmptyListCheck of ProjectionStep list + | TupleArityCheck of ProjectionStep list * int + | RecordFieldCheck of ProjectionStep list * string + | SomeCheck of ProjectionStep list + | NoneCheck of ProjectionStep list + | UnionCaseCheck of ProjectionStep list * string * string * bool + | TypeCheck of ProjectionStep list * Type + + and PatternBinding = + { Id: SymbolId + Name: string + Type: Type + Projection: ProjectionStep list } + + and PatternPlan = + { Span: Span + Checks: PatternCheck list + Bindings: PatternBinding list + Maps: MapPlan list } + + and MapPlan = + { Projection: ProjectionStep list + Clauses: MapClausePlan list + Tail: PatternPlan option + RequireEmptyRemainder: bool } + + and MapClausePlan = + { Key: PatternPlan + Value: PatternPlan } and DecisionCase = - { Pattern: Pattern + { Plan: PatternPlan Guard: Expression option Body: Expression Span: Span } @@ -153,6 +176,7 @@ module Lir = | Load id -> add $"load %%{symbolNumber id}" | MakeClosure(id, captures) -> add $"make-closure @{functionNumber id} captures={symbols captures}" | CallClosure(fn, argument) -> children "call-closure" [ fn; argument ] + | PipeClosure(value, fn) -> children "pipe-closure" [ value; fn ] | Branch(condition, whenTrue, whenFalse) -> children "branch" [ condition; whenTrue; whenFalse ] | Operation(operation, arguments) -> children $"operation {operation}" arguments | Tuple items -> children "tuple" items @@ -221,43 +245,74 @@ module LirLowering = | Hir.Unit -> Lir.Unit | Hir.Int value -> Lir.Int value | Hir.Float value -> Lir.Float value | Hir.Bool value -> Lir.Bool value | Hir.String value -> Lir.String value - let rec private pattern (value: Hir.Pattern) : Lir.Pattern = - let node = - match value.Node with - | Hir.Wildcard -> Lir.Wildcard - | Hir.Binding(id, name) -> Lir.Binding(symbol id, name) - | Hir.LiteralPattern value -> Lir.LiteralPattern(constant value) - | Hir.EmptyList -> Lir.EmptyList - | Hir.ConsPattern(head, tail) -> Lir.ConsPattern(pattern head, pattern tail) - | Hir.TuplePattern items -> Lir.TuplePattern(List.map pattern items) - | Hir.RecordPattern fields -> Lir.RecordPattern(fields |> List.map (fun (name, item) -> name, pattern item)) - | Hir.MapPattern(clauses, tail) -> Lir.MapPattern(clauses |> List.map (fun (key, value) -> pattern key, pattern value), Option.map pattern tail) - | Hir.SomePattern item -> Lir.SomePattern(pattern item) - | Hir.NonePattern -> Lir.NonePattern - | Hir.UnionPattern(typeName, caseName, payload) -> Lir.UnionPattern(typeName, caseName, Option.map pattern payload) - | Hir.TypePattern patternType -> Lir.TypePattern patternType - { Type = value.Type; Span = value.Span; Node = node } + let private appendPlan (left: Lir.PatternPlan) (right: Lir.PatternPlan) : Lir.PatternPlan = + { Span = left.Span + Checks = left.Checks @ right.Checks + Bindings = left.Bindings @ right.Bindings + Maps = left.Maps @ right.Maps } - let rec private patternBindings (value: Lir.Pattern) = + let rec private patternPlan projection (value: Hir.Pattern) : Lir.PatternPlan = + let empty () : Lir.PatternPlan = { Span = value.Span; Checks = []; Bindings = []; Maps = [] } + let combine plans = plans |> List.fold appendPlan (empty ()) match value.Node with - | Lir.Binding(id, _) -> Set.singleton id - | Lir.ConsPattern(head, tail) -> Set.union (patternBindings head) (patternBindings tail) - | Lir.TuplePattern items -> items |> List.map patternBindings |> Set.unionMany - | Lir.RecordPattern fields -> fields |> List.map (snd >> patternBindings) |> Set.unionMany - | Lir.MapPattern(clauses, tail) -> - [ yield! clauses |> List.collect (fun (key, value) -> [ patternBindings key; patternBindings value ]) - tail |> Option.map patternBindings |> Option.defaultValue Set.empty ] - |> Set.unionMany - | Lir.SomePattern item -> patternBindings item - | Lir.UnionPattern(_, _, payload) -> payload |> Option.map patternBindings |> Option.defaultValue Set.empty - | _ -> Set.empty + | Hir.Wildcard -> empty () + | Hir.Binding(id, name) -> + { empty () with + Bindings = [ { Id = symbol id; Name = name; Type = value.Type; Projection = projection } ] } + | Hir.LiteralPattern item -> { empty () with Checks = [ Lir.LiteralCheck(projection, constant item) ] } + | Hir.EmptyList -> { empty () with Checks = [ Lir.EmptyListCheck projection ] } + | Hir.ConsPattern(head, tail) -> + combine + [ { empty () with Checks = [ Lir.NonEmptyListCheck projection ] } + patternPlan (projection @ [ Lir.ListHead ]) head + patternPlan (projection @ [ Lir.ListTail ]) tail ] + | Hir.TuplePattern items -> + { empty () with Checks = [ Lir.TupleArityCheck(projection, items.Length) ] } + :: (items |> List.mapi (fun index item -> patternPlan (projection @ [ Lir.TupleItem index ]) item)) + |> combine + | Hir.RecordPattern fields -> + fields + |> List.collect (fun (name, item) -> + [ { empty () with Checks = [ Lir.RecordFieldCheck(projection, name) ] } + patternPlan (projection @ [ Lir.RecordField name ]) item ]) + |> combine + | Hir.MapPattern(clauses, tail) -> + let mapPlan : Lir.MapPlan = + { Projection = projection + Clauses = + clauses + |> List.map (fun (key, item) -> + { Key = patternPlan [] key + Value = patternPlan [] item }) + Tail = Option.map (patternPlan []) tail + RequireEmptyRemainder = clauses.IsEmpty && tail.IsNone } + { empty () with Maps = [ mapPlan ] } + | Hir.SomePattern item -> + combine + [ { empty () with Checks = [ Lir.SomeCheck projection ] } + patternPlan (projection @ [ Lir.OptionValue ]) item ] + | Hir.NonePattern -> { empty () with Checks = [ Lir.NoneCheck projection ] } + | Hir.UnionPattern(typeName, caseName, payload) -> + combine + [ { empty () with Checks = [ Lir.UnionCaseCheck(projection, typeName, caseName, payload.IsSome) ] } + yield! payload |> Option.map (patternPlan (projection @ [ Lir.UnionPayload ])) |> Option.toList ] + | Hir.TypePattern patternType -> { empty () with Checks = [ Lir.TypeCheck(projection, patternType) ] } + + let rec private patternBindings (plan: Lir.PatternPlan) = + [ plan.Bindings |> List.map _.Id |> Set.ofList + yield! + plan.Maps + |> List.collect (fun mapPlan -> + [ yield! mapPlan.Clauses |> List.collect (fun clause -> [ patternBindings clause.Key; patternBindings clause.Value ]) + yield! mapPlan.Tail |> Option.map patternBindings |> Option.toList ]) ] + |> Set.unionMany let rec capturedReferences bound (value: Lir.Expression) = let free = capturedReferences bound match value.Node with | Lir.Load id -> if Set.contains id bound then Set.empty else Set.singleton id | Lir.MakeClosure(_, captures) -> Set.difference (Set.ofList captures) bound - | Lir.CallClosure(fn, argument) | Lir.Append(fn, argument) | Lir.Cons(fn, argument) + | Lir.CallClosure(fn, argument) | Lir.PipeClosure(argument, fn) | Lir.Append(fn, argument) | Lir.Cons(fn, argument) | Lir.Range(fn, argument) | Lir.IndexGet(fn, argument) -> Set.union (free fn) (free argument) | Lir.Branch(condition, whenTrue, whenFalse) -> [ free condition; free whenTrue; free whenFalse ] |> Set.unionMany | Lir.Operation(_, arguments) | Lir.Tuple arguments | Lir.List arguments -> arguments |> List.map free |> Set.unionMany @@ -272,7 +327,7 @@ module LirLowering = | Lir.DecisionTree(target, cases) -> free target :: (cases |> List.collect (fun case -> - let scope = Set.union bound (patternBindings case.Pattern) + let scope = Set.union bound (patternBindings case.Plan) [ yield! case.Guard |> Option.map (capturedReferences scope) |> Option.toList capturedReferences scope case.Body ])) |> Set.unionMany @@ -299,6 +354,7 @@ module LirLowering = Captures = captures; ReturnType = loweredBody.Type; Span = value.Span; Body = loweredBody } Lir.MakeClosure(functionId, captures) | Hir.Apply(fn, argument) -> Lir.CallClosure(expression state fn, expression state argument) + | Hir.Pipe(argument, fn) -> Lir.PipeClosure(expression state argument, expression state fn) | Hir.If(condition, whenTrue, whenFalse) -> Lir.Branch(expression state condition, expression state whenTrue, expression state whenFalse) | Hir.Primitive(op, arguments) -> Lir.Operation(operation op, arguments |> List.map (expression state)) | Hir.Tuple items -> Lir.Tuple(items |> List.map (expression state)) @@ -315,7 +371,7 @@ module LirLowering = | Hir.Fail value -> Lir.Fail(expression state value) | Hir.Match(target, cases) -> Lir.DecisionTree(expression state target, cases |> List.map (fun case -> - ({ Pattern = pattern case.Pattern + ({ Plan = patternPlan [] case.Pattern Guard = Option.map (expression state) case.Guard Body = expression state case.Body Span = case.Span }: Lir.DecisionCase))) @@ -421,51 +477,52 @@ module LirValidation = program.Declarations |> List.iter (fun declaration -> if declaration.IsExported <> declaration.Export.IsSome || (declaration.Export |> Option.exists (fun export -> export.Name <> declaration.Name || not (validSpan export.Span))) then report "FSL2015" "LIR export metadata is inconsistent." declaration.Span) - let rec checkPattern expected (pattern: Lir.Pattern) = - checkSpan pattern.Span - if not (compatible expected pattern.Type) then report "FSL2003" "LIR pattern type does not match its scrutinee." pattern.Span - match pattern.Node with - | Lir.LiteralPattern value when not (compatible pattern.Type (literalType value)) -> report "FSL2003" "LIR literal pattern has an inconsistent type." pattern.Span - | Lir.ConsPattern(head, tail) -> - match pattern.Type with - | TList item -> checkPattern item head; checkPattern pattern.Type tail - | _ -> report "FSL2003" "LIR cons pattern requires a list." pattern.Span - | Lir.TuplePattern items -> - match pattern.Type with - | TTuple types when types.Length = items.Length -> List.iter2 checkPattern types items - | _ -> report "FSL2003" "LIR tuple pattern shape is inconsistent." pattern.Span - | Lir.RecordPattern fields -> - match resolve pattern.Type with - | TRecord types -> fields |> List.iter (fun (name, item) -> match Map.tryFind name types with Some itemType -> checkPattern itemType item | None -> report "FSL2003" $"Unknown LIR record-pattern field '{name}'." item.Span) - | _ -> report "FSL2003" "LIR record pattern requires a record." pattern.Span - | Lir.SomePattern item -> match pattern.Type with TOption itemType -> checkPattern itemType item | _ -> report "FSL2003" "LIR some pattern requires an option." pattern.Span - | Lir.EmptyList -> match pattern.Type with TList _ -> () | _ -> report "FSL2003" "LIR empty-list pattern requires a list." pattern.Span - | Lir.NonePattern -> match pattern.Type with TOption _ -> () | _ -> report "FSL2003" "LIR none pattern requires an option." pattern.Span - | Lir.MapPattern(clauses, tail) -> - match pattern.Type with - | TMap(keyType, valueType) -> clauses |> List.iter (fun (key, value) -> checkPattern keyType key; checkPattern valueType value); tail |> Option.iter (checkPattern pattern.Type) - | _ -> report "FSL2003" "LIR map pattern requires a map." pattern.Span - | Lir.UnionPattern(typeName, caseName, payload) -> - match resolve pattern.Type with - | TUnion(actual, cases) when actual = typeName -> - match Map.tryFind caseName cases, payload with - | Some(Some payloadType), Some value -> checkPattern payloadType value - | Some None, None -> () - | _ -> report "FSL2003" "LIR union-pattern payload is inconsistent." pattern.Span - | _ -> report "FSL2003" "LIR union pattern has an inconsistent declared type." pattern.Span - | _ -> () - let rec patternBindings (pattern: Lir.Pattern) = - match pattern.Node with - | Lir.Binding(id, _) -> Set.singleton id - | Lir.ConsPattern(head, tail) -> Set.union (patternBindings head) (patternBindings tail) - | Lir.TuplePattern items -> items |> List.map patternBindings |> Set.unionMany - | Lir.RecordPattern fields -> fields |> List.map (snd >> patternBindings) |> Set.unionMany - | Lir.MapPattern(clauses, tail) -> - [ yield! clauses |> List.collect (fun (key, value) -> [ patternBindings key; patternBindings value ]) - match tail with Some value -> patternBindings value | None -> Set.empty ] |> Set.unionMany - | Lir.SomePattern value -> patternBindings value - | Lir.UnionPattern(_, _, payload) -> payload |> Option.map patternBindings |> Option.defaultValue Set.empty - | _ -> Set.empty + let rec projectionType current = function + | [] -> Some current + | step :: rest -> + let next = + match step, resolve current with + | Lir.ListHead, TList item -> Some item + | Lir.ListTail, TList _ -> Some current + | Lir.TupleItem index, TTuple items when index >= 0 && index < items.Length -> Some items.[index] + | Lir.RecordField name, TRecord fields -> Map.tryFind name fields + | Lir.OptionValue, TOption item -> Some item + // The preceding union-case check selects the concrete + // payload. Binding/check types validate that selection. + | Lir.UnionPayload, TUnion _ -> Some(TVar -1) + | _ -> None + next |> Option.bind (fun item -> projectionType item rest) + let rec checkPlan expected (plan: Lir.PatternPlan) = + checkSpan plan.Span + let at projection = projectionType expected projection + let require projection predicate message = + match at projection with Some value when predicate (resolve value) -> () | _ -> report "FSL2003" message plan.Span + for check in plan.Checks do + match check with + | Lir.LiteralCheck(projection, value) -> require projection (compatible (literalType value)) "LIR literal plan has an inconsistent projection type." + | Lir.EmptyListCheck projection | Lir.NonEmptyListCheck projection -> require projection (function TList _ -> true | _ -> false) "LIR list plan requires a list projection." + | Lir.TupleArityCheck(projection, arity) -> require projection (function TTuple items -> items.Length = arity | _ -> false) "LIR tuple plan has an inconsistent arity." + | Lir.RecordFieldCheck(projection, name) -> require projection (function TRecord fields -> Map.containsKey name fields | _ -> false) $"LIR record plan references unknown field '{name}'." + | Lir.SomeCheck projection | Lir.NoneCheck projection -> require projection (function TOption _ -> true | _ -> false) "LIR option plan requires an option projection." + | Lir.UnionCaseCheck(projection, typeName, caseName, hasPayload) -> + require projection (function TUnion(actual, cases) when actual = typeName -> Map.tryFind caseName cases |> Option.exists (fun payload -> payload.IsSome = hasPayload) | _ -> false) "LIR union plan has inconsistent case metadata." + | Lir.TypeCheck(projection, _) -> require projection (fun _ -> true) "LIR type-test plan has an invalid projection." + for binding in plan.Bindings do + require binding.Projection (compatible binding.Type) "LIR pattern binding has an inconsistent projection type." + for mapPlan in plan.Maps do + match at mapPlan.Projection |> Option.map resolve with + | Some(TMap(keyType, valueType) as mapType) -> + mapPlan.Clauses |> List.iter (fun clause -> checkPlan keyType clause.Key; checkPlan valueType clause.Value) + mapPlan.Tail |> Option.iter (checkPlan mapType) + | _ -> report "FSL2003" "LIR map plan requires a map projection." plan.Span + let rec patternBindings (plan: Lir.PatternPlan) = + [ plan.Bindings |> List.map _.Id |> Set.ofList + yield! + plan.Maps + |> List.collect (fun mapPlan -> + [ yield! mapPlan.Clauses |> List.collect (fun clause -> [ patternBindings clause.Key; patternBindings clause.Value ]) + yield! mapPlan.Tail |> Option.map patternBindings |> Option.toList ]) ] + |> Set.unionMany let declared = ids |> Set.ofList let rec checkExpression locals (expression: Lir.Expression) = checkSpan expression.Span @@ -486,6 +543,9 @@ module LirValidation = | Lir.CallClosure(fn, argument) -> check fn; check argument match fn.Type with TFun(parameter, result) when compatible parameter argument.Type && compatible result expression.Type -> () | _ -> report "FSL2006" "LIR closure call signature is inconsistent." expression.Span + | Lir.PipeClosure(argument, fn) -> + check argument; check fn + match fn.Type with TFun(parameter, result) when compatible parameter argument.Type && compatible result expression.Type -> () | _ -> report "FSL2006" "LIR pipeline call signature is inconsistent." expression.Span | Lir.Branch(condition, whenTrue, whenFalse) -> check condition; check whenTrue; check whenFalse if not (compatible condition.Type TBool && compatible expression.Type whenTrue.Type && compatible expression.Type whenFalse.Type) then report "FSL2007" "LIR branch types are inconsistent." expression.Span @@ -517,7 +577,7 @@ module LirValidation = | Lir.Let(id, _, recursive, binding, body) -> let scope = Set.add id locals in checkExpression (if recursive then scope else locals) binding; checkExpression scope body; if not (compatible expression.Type body.Type) then report "FSL2004" "LIR let result type is inconsistent." expression.Span | Lir.LetRecursiveGroup(bindings, body) -> let scope = bindings |> List.fold (fun value (id, _, _, _) -> Set.add id value) locals in bindings |> List.iter (fun (_, _, bindingType, value) -> checkExpression scope value; if not (compatible bindingType value.Type) then report "FSL2006" "LIR recursive binding signature is inconsistent." value.Span); checkExpression scope body; if not (compatible expression.Type body.Type) then report "FSL2004" "LIR recursive group result is inconsistent." expression.Span | Lir.Fail value -> check value - | Lir.DecisionTree(target, cases) -> check target; cases |> List.iter (fun case -> let caseLocals = Set.union locals (patternBindings case.Pattern) in checkPattern target.Type case.Pattern; case.Guard |> Option.iter (fun guard -> checkExpression caseLocals guard; if not (compatible guard.Type TBool) then report "FSL2007" "LIR decision guard must be Boolean." guard.Span); checkExpression caseLocals case.Body; if not (compatible expression.Type case.Body.Type) then report "FSL2007" "LIR decision body type is inconsistent." case.Body.Span) + | Lir.DecisionTree(target, cases) -> check target; cases |> List.iter (fun case -> let caseLocals = Set.union locals (patternBindings case.Plan) in checkPlan target.Type case.Plan; case.Guard |> Option.iter (fun guard -> checkExpression caseLocals guard; if not (compatible guard.Type TBool) then report "FSL2007" "LIR decision guard must be Boolean." guard.Span); checkExpression caseLocals case.Body; if not (compatible expression.Type case.Body.Type) then report "FSL2007" "LIR decision body type is inconsistent." case.Body.Span) | Lir.Range(first, last) -> check first; check last; if not (compatible first.Type TInt && compatible last.Type TInt && compatible expression.Type (TList TInt)) then report "FSL2004" "LIR range representation is inconsistent." expression.Span | Lir.RecordUpdate(target, fields) -> check target; fields |> List.iter (snd >> check); if not (compatible expression.Type target.Type) then report "FSL2004" "LIR record update representation is inconsistent." expression.Span | Lir.RecordConvert(target, targetType) -> check target; match resolve target.Type, resolve targetType with TRecord _, TRecord _ when compatible expression.Type targetType -> () | _ -> report "FSL2004" "LIR record conversion evidence is inconsistent." expression.Span diff --git a/src/FScript.Interpreter.Lir/LirInterpreter.fs b/src/FScript.Interpreter.Lir/LirInterpreter.fs index ed73643..4a3f140 100644 --- a/src/FScript.Interpreter.Lir/LirInterpreter.fs +++ b/src/FScript.Interpreter.Lir/LirInterpreter.fs @@ -196,16 +196,7 @@ module LirInterpreter = | "Map.remove", [ key; VMap entries ] -> Ok(VMap(entries |> List.filter (fst >> equal key >> not))) | _ -> unsupported () - let rec private matchPattern (pattern: Lir.Pattern) value = - let combine left right = - match left, right with - | Some a, Some b -> - b |> Map.fold (fun state key item -> - state |> Option.bind (fun env -> - match env |> Map.tryFind key with - | Some existing when not (equal existing item) -> None - | _ -> Some(Map.add key item env))) (Some a) - | _ -> None + let rec private matchPlan (plan: Lir.PatternPlan) value = let rec matchesType expected item = match expected, item with | TUnit, VUnit | TInt, VInt _ | TFloat, VFloat _ | TBool, VBool _ | TString, VString _ | TTypeToken, VTypeToken _ -> true @@ -220,39 +211,66 @@ module LirInterpreter = | TFun _, (VClosure _ | VIntrinsic _ | VExternal _) -> true | TVar _, _ -> true | _ -> false - match pattern.Node, value with - | Lir.Wildcard, _ -> Some Map.empty - | Lir.Binding(id, _), item -> Some(Map.ofList [ id, item ]) - | Lir.LiteralPattern literal, item when equal (constantValue literal) item -> Some Map.empty - | Lir.EmptyList, VList [] -> Some Map.empty - | Lir.ConsPattern(head, tail), VList(headValue :: tailValues) -> - combine (matchPattern head headValue) (matchPattern tail (VList tailValues)) - | Lir.TuplePattern patterns, VTuple values when patterns.Length = values.Length -> - List.zip patterns values |> List.fold (fun state (itemPattern, itemValue) -> combine state (matchPattern itemPattern itemValue)) (Some Map.empty) - | Lir.RecordPattern patterns, VRecord values -> - patterns |> List.fold (fun state (name, itemPattern) -> - combine state (values |> Map.tryFind name |> Option.bind (matchPattern itemPattern))) (Some Map.empty) - | Lir.MapPattern(clauses, tailPattern), VMap entries -> - let rec matchClauses bindings remaining = function - | [] -> - match tailPattern with - | Some tail -> combine (Some bindings) (matchPattern tail (VMap remaining)) - | None when not clauses.IsEmpty || remaining.IsEmpty -> Some bindings - | None -> None - | (keyPattern, valuePattern) :: rest -> - remaining |> List.indexed |> List.tryPick (fun (index, (key, item)) -> - combine (Some bindings) (matchPattern keyPattern key) - |> Option.bind (fun withKey -> combine (Some withKey) (matchPattern valuePattern item)) - |> Option.bind (fun withValue -> - let next = remaining |> List.indexed |> List.choose (fun (position, entry) -> if position = index then None else Some entry) - matchClauses withValue next rest)) - matchClauses Map.empty entries clauses - | Lir.SomePattern itemPattern, VOption(Some item) -> matchPattern itemPattern item - | Lir.NonePattern, VOption None -> Some Map.empty - | Lir.UnionPattern(typeName, caseName, payloadPattern), VUnion(valueType, valueCase, payload) when typeName = valueType && caseName = valueCase -> - match payloadPattern, payload with None, None -> Some Map.empty | Some pattern, Some value -> matchPattern pattern value | _ -> None - | Lir.TypePattern expected, item when matchesType expected item -> Some Map.empty - | _ -> None + let rec project (item: LirValue) (projection: Lir.ProjectionStep list) = + match projection with + | [] -> Some item + | step :: rest -> + let next = + match step, item with + | Lir.ListHead, VList(head :: _) -> Some head + | Lir.ListTail, VList(_ :: tail) -> Some(VList tail) + | Lir.TupleItem index, VTuple items when index >= 0 && index < items.Length -> Some items.[index] + | Lir.RecordField name, VRecord fields -> Map.tryFind name fields + | Lir.OptionValue, VOption(Some inner) -> Some inner + | Lir.UnionPayload, VUnion(_, _, Some inner) -> Some inner + | _ -> None + next |> Option.bind (fun inner -> project inner rest) + let addBinding (bindings: Map) id item = + match Map.tryFind id bindings with + | Some existing when not (equal existing item) -> None + | _ -> Some(Map.add id item bindings) + let rec run (currentPlan: Lir.PatternPlan) (root: LirValue) (bindings: Map) = + if not (currentPlan.Checks |> List.forall (fun item -> + // Checks are evaluated against the plan's current root. + match item with + | Lir.LiteralCheck(projection, literal) -> project root projection |> Option.exists (equal (constantValue literal)) + | Lir.EmptyListCheck projection -> project root projection |> Option.exists (function VList [] -> true | _ -> false) + | Lir.NonEmptyListCheck projection -> project root projection |> Option.exists (function VList(_ :: _) -> true | _ -> false) + | Lir.TupleArityCheck(projection, arity) -> project root projection |> Option.exists (function VTuple items -> items.Length = arity | _ -> false) + | Lir.RecordFieldCheck(projection, name) -> project root projection |> Option.exists (function VRecord fields -> Map.containsKey name fields | _ -> false) + | Lir.SomeCheck projection -> project root projection |> Option.exists (function VOption(Some _) -> true | _ -> false) + | Lir.NoneCheck projection -> project root projection |> Option.exists (function VOption None -> true | _ -> false) + | Lir.UnionCaseCheck(projection, typeName, caseName, hasPayload) -> project root projection |> Option.exists (function VUnion(actualType, actualCase, payload) -> actualType = typeName && actualCase = caseName && payload.IsSome = hasPayload | _ -> false) + | Lir.TypeCheck(projection, expected) -> project root projection |> Option.exists (matchesType expected))) then None + else + let withBindings = + currentPlan.Bindings + |> List.fold (fun state binding -> + state |> Option.bind (fun env -> project root binding.Projection |> Option.bind (addBinding env binding.Id))) (Some bindings) + let runMap env (mapPlan: Lir.MapPlan) = + project root mapPlan.Projection + |> Option.bind (function + | VMap entries -> + let rec clauses current remaining (pending: Lir.MapClausePlan list) = + match pending with + | [] -> + match mapPlan.Tail with + | Some tail -> run tail (VMap remaining) current + | None when not mapPlan.RequireEmptyRemainder || remaining.IsEmpty -> Some current + | None -> None + | clause :: rest -> + remaining + |> List.indexed + |> List.tryPick (fun (index, (key, item)) -> + run clause.Key key current + |> Option.bind (fun withKey -> run clause.Value item withKey) + |> Option.bind (fun withValue -> + let next = remaining |> List.indexed |> List.choose (fun (position, entry) -> if position = index then None else Some entry) + clauses withValue next rest)) + clauses env entries mapPlan.Clauses + | _ -> None) + currentPlan.Maps |> List.fold (fun state mapPlan -> state |> Option.bind (fun env -> runMap env mapPlan)) withBindings + run plan value Map.empty let rec private evaluate environment (expression: Lir.Expression) = let binary work = @@ -276,6 +294,9 @@ module LirInterpreter = | Lir.CallClosure(fn, argument) -> evaluate environment fn |> Result.bind (fun fnValue -> evaluate environment argument |> Result.bind (applyValue expression.Span fnValue)) + | Lir.PipeClosure(argument, fn) -> + evaluate environment argument + |> Result.bind (fun argumentValue -> evaluate environment fn |> Result.bind (fun fnValue -> applyValue expression.Span fnValue argumentValue)) | Lir.Branch(condition, whenTrue, whenFalse) -> match evaluate environment condition with | Ok(VBool true) -> evaluate environment whenTrue @@ -360,7 +381,7 @@ module LirInterpreter = match remaining with | [] -> fail "FSI3012" expression.Span "Match failure." | case :: rest -> - match matchPattern case.Pattern targetValue with + match matchPlan case.Plan targetValue with | None -> run rest | Some bindings -> let caseEnvironment = Map.fold (fun env id value -> Map.add id value env) environment bindings diff --git a/src/FScript.Language/TypeInfer.fs b/src/FScript.Language/TypeInfer.fs index 4fa8a44..96d97f7 100644 --- a/src/FScript.Language/TypeInfer.fs +++ b/src/FScript.Language/TypeInfer.fs @@ -229,6 +229,7 @@ module TypeInfer = | ResolvedReference of ResolvedSymbolReference | ResolvedLambda of ResolvedBinding * ResolvedExpression | ResolvedApply of ResolvedExpression * ResolvedExpression + | ResolvedPipe of ResolvedExpression * ResolvedExpression | ResolvedIf of ResolvedExpression * ResolvedExpression * ResolvedExpression | ResolvedPrimitive of ResolvedPrimitiveOperation * ResolvedExpression list | ResolvedTuple of ResolvedExpression list @@ -1895,6 +1896,8 @@ module TypeInfer = ResolvedLambda(bindingAt ParameterSymbol parameter.Name parameter.Span parameterType, resolveExpression body) | EApply(fn, argument, _) -> ResolvedApply(resolveExpression fn, resolveExpression argument) | EIf(condition, whenTrue, whenFalse, _) -> ResolvedIf(resolveExpression condition, resolveExpression whenTrue, resolveExpression whenFalse) + | EBinOp("|>", left, right, _) -> ResolvedPipe(resolveExpression left, resolveExpression right) + | EBinOp("::", head, tail, _) -> ResolvedCons(resolveExpression head, resolveExpression tail) | EBinOp("@", left, right, _) | EAppend(left, right, _) -> ResolvedAppend(resolveExpression left, resolveExpression right) | EBinOp(_, left, right, _) -> ResolvedPrimitive(Option.get primitive, [ resolveExpression left; resolveExpression right ]) | ECons(head, tail, _) -> ResolvedCons(resolveExpression head, resolveExpression tail) @@ -1951,7 +1954,7 @@ module TypeInfer = let children = match node with | ResolvedLambda(_, body) | ResolvedSome body | ResolvedFail body -> [ body ] - | ResolvedApply(left, right) | ResolvedCons(left, right) | ResolvedAppend(left, right) + | ResolvedApply(left, right) | ResolvedPipe(left, right) | ResolvedCons(left, right) | ResolvedAppend(left, right) | ResolvedRange(left, right) | ResolvedIndexGet(left, right) -> [ left; right ] | ResolvedIf(condition, whenTrue, whenFalse) -> [ condition; whenTrue; whenFalse ] | ResolvedPrimitive(_, values) | ResolvedTuple values | ResolvedList values -> values @@ -1972,6 +1975,11 @@ module TypeInfer = | TFun((TNamed target as targetType), _), (TRecord _ as sourceType) -> Some { SourceType = sourceType; TargetType = targetType; TargetName = Some target } | TFun((TRecord _ as targetType), _), (TNamed source as sourceType) -> Some { SourceType = sourceType; TargetType = targetType; TargetName = Some source } | _ -> None + | EBinOp("|>", argument, fn, _) -> + match finalType fn, finalType argument with + | TFun((TNamed target as targetType), _), (TRecord _ as sourceType) -> Some { SourceType = sourceType; TargetType = targetType; TargetName = Some target } + | TFun((TRecord _ as targetType), _), (TNamed source as sourceType) -> Some { SourceType = sourceType; TargetType = targetType; TargetName = Some source } + | _ -> None | _ -> None { Type = expressionType; Span = Ast.spanOfExpr expression; Reference = reference Primitive = primitive; RecordCoercion = coercion; Node = node; Children = children } diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index 768a1a8..a31c702 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -132,7 +132,8 @@ type GoBackendTests() = | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) | Ok files -> let program = files |> List.find (fun file -> file.RelativePath.EndsWith("program.go")) |> _.Content - Assert.That(program, Does.Contain("fsruntime.MatchPattern(matchValue")) + Assert.That(program, Does.Contain("fsruntime.MatchPatternPlan(matchValue")) + Assert.That(program, Does.Contain("fsruntime.ProjectionStep{Kind: \"option-value\"}")) Assert.That(program, Does.Not.Contain("fsruntime.OptionValue(matchValue")) Assert.That(program, Does.Not.Contain("fsruntime.AsTuple(matchValue")) @@ -172,6 +173,7 @@ type GoBackendTests() = [] [ 1; \"b\" => 2 }")>] [ 1; \"b\" => 2 }")>] + [ String.concat \",\"", "\"a,b\"")>] [ value + 1) (Map.filter (fun key value -> value > 1) (Map.ofList [(\"b\", 2); (\"a\", 1)]))", "map { \"b\" => 3 }")>] [ if value > 1 then Some (value + 1) else None) { [\"a\"] = 1; [\"b\"] = 2 }", "map { \"b\" => 3 }")>] [ 2 }")>] diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index 8f33487..52bf189 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -183,3 +183,22 @@ type HirTests() = let expression = ({ Type = TInt; Span = span; Node = Lir.Constant(Lir.Int 1L) }: Lir.Expression) let program: Lir.Program = { Types = []; ImportedRecords = []; Functions = []; Declarations = []; Entry = Some expression } Assert.That(LirOptimization.optimize program = program, Is.True) + + [] + member _.``LIR lowers patterns into projection and check plans`` () = + let compiled = + match Compiler.compile { Program = Parser.parseProgram "match Some (1, 2) with | Some (left, right) -> left + right | None -> 0"; Externs = [] } with + | Ok value -> value + | Error diagnostics -> failwith (System.String.Join("; ", diagnostics |> List.map _.Message)) + let decision = + compiled.Lir.Declarations + |> List.choose (fun declaration -> match declaration.Value.Node with Lir.DecisionTree(_, cases) -> Some cases | _ -> None) + |> List.exactlyOne + let first = decision.Head.Plan + Assert.That(first.Checks |> List.exists (function Lir.SomeCheck [] -> true | _ -> false), Is.True) + Assert.That(first.Checks |> List.exists (function Lir.TupleArityCheck([ Lir.OptionValue ], 2) -> true | _ -> false), Is.True) + let projections = first.Bindings |> List.map _.Projection + let expected = + [ [ Lir.OptionValue; Lir.TupleItem 0 ] + [ Lir.OptionValue; Lir.TupleItem 1 ] ] + Assert.That((projections = expected), Is.True) diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index ca0c022..96bd804 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -109,12 +109,21 @@ type IrArtifactTests() = "match { [\"a\"] = 1 } with\n | { [key] = value; ..tail } -> value\n | {} -> 0" "match {| Name = \"Ada\" |} with\n | {| Name: string |} -> true\n | _ -> false" "List.length Env.Arguments" + "[\"a\"; \"b\"] |> String.concat \",\"" "[] let upper : string -> string\nupper \"hello\"" ] do let bytes = compile source |> _.Lir |> IrArtifact.serialize match IrArtifact.deserialize bytes with | Ok _ -> () | Error diagnostics -> Assert.Fail(String.Join("; ", diagnostics |> List.map _.Message)) + [] + member _.``IR artifact serializes lowered pattern plans rather than source patterns`` () = + let bytes = compile "match Some (1, 2) with | Some (left, right) -> left + right | None -> 0" |> _.Lir |> IrArtifact.serialize + let json = Encoding.UTF8.GetString(bytes) + Assert.That(json, Does.Contain("\"plan\"")) + Assert.That(json, Does.Contain("\"projection\"")) + Assert.That(json, Does.Not.Contain("\"pattern\"")) + [] member _.``IR artifact preserves explicit structural record conversions`` () = let source = "type Person = { Name: string; Age: int }\nlet age (person: Person) = person.Age\nage {| Name = \"Ada\"; Age = 42 |}" diff --git a/tests/FScript.Conformance.Tests/ConformanceTests.fs b/tests/FScript.Conformance.Tests/ConformanceTests.fs index a0b2c23..bbf920b 100644 --- a/tests/FScript.Conformance.Tests/ConformanceTests.fs +++ b/tests/FScript.Conformance.Tests/ConformanceTests.fs @@ -121,6 +121,7 @@ type ConformanceTests() = [ 8) None, Option.isNone None, Option.isSome (Some 1), Option.map (fun value -> value + 1) (Some 1))")>] [ state + value) 0 (Map.ofList [(\"a\", 1); (\"b\", 2)]), Map.filter (fun key value -> value > 1) (Map.ofList [(\"a\", 1); (\"b\", 2)]), Map.choose (fun key value -> if value > 1 then Some value else None) (Map.ofList [(\"a\", 1); (\"b\", 2)]), Map.map (fun value -> value + 1) (Map.ofList [(\"a\", 1)]), Map.iter (fun key value -> ignore value) (Map.ofList [(\"a\", 1)]), Map.remove \"a\" (Map.ofList [(\"a\", 1); (\"b\", 2)]))")>] [] + [ String.concat \",\"")>] member _.``Legacy LIR artifact and Go results conform`` (source: string) = let expected = Harness.legacy source match Harness.lir source with From d90b73bfe4a7ce965fca35ea081cab0670339b1f Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 22:04:20 +0200 Subject: [PATCH 060/133] fix(compiler): compile complete sample semantics --- CHANGELOG.md | 2 + docs/architecture/typed-ir-and-backends.md | 2 + docs/specs/compiler-semantics-baseline.md | 2 + src/FScript.Backend.Go/GoBackend.fs | 6 ++- src/FScript.Compiler/HirLowering.fs | 52 ++++++++++++++----- src/FScript.Compiler/HirValidation.fs | 7 ++- src/FScript.Compiler/Lir.fs | 52 ++++++++++++++++--- src/FScript.Language/Lexer.fs | 9 ++-- src/FScript.Language/Parser.fs | 21 ++++++-- src/FScript.Language/TypeInfer.fs | 40 ++++++++++---- tests/FScript.Compiler.Tests/HirTests.fs | 40 +++++++++++--- .../FScript.Compiler.Tests/Snapshots/add.hir | 16 +++--- .../FScript.Compiler.Tests/Snapshots/add.lir | 18 +++---- tests/FScript.Language.Tests/ParserTests.fs | 11 ++++ .../ResolvedInferenceTests.fs | 36 +++++++++++++ 15 files changed, 249 insertions(+), 65 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 962f8b4..c2af80e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Compiled every shipped sample by preserving interpolation scopes, intrinsic types, recursive representations, and structural record-width conversions. + - Lowered source patterns into backend-neutral executable LIR projection and decision plans. - Preserved value-first pipeline evaluation through resolved inference and compiled backends. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 1804176..dc46b12 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -32,6 +32,8 @@ HIR and LIR expose canonical debug printers with stable artifact-local IDs and n HIR declaration schemes are taken directly from the resolved frontend after deterministic type-variable normalization; lowering does not generalize source declarations again. Synthetic declarations may derive a local scheme because they have no frontend declaration counterpart. +Compiler artifacts are closed over the language's intrinsic `Environment` record and `FsKind` union definitions. Those types and their cases receive deterministic frontend symbol IDs just like source declarations; an intrinsic constructor value is emitted only when referenced. LIR compatibility and pattern validation resolve named types recursively with cycle guards, including concrete payload types selected by a preceding union-case check. + Resolved statements expose normalized structured attributes as name, argument list, and source span. The parser retains unknown attributes instead of assigning semantics to them, and HIR attaches them to ordinary value and type declarations while retaining dedicated export and target-import linkage for consumers that need semantic access without reparsing attribute arguments. HIR validation rejects duplicate and unresolved symbols, invalid declaration spans, and schemes inconsistent with their values. LIR validation rechecks artifact-local references and spans, then verifies closure calls, branches and decision guards, primitive signatures, aggregate representations, declaration value types, intrinsic catalog membership, and complete target-import linkage. Artifact deserialization, LIR execution, and Go generation run the same validator, so malformed or directly constructed LIR cannot bypass compiler invariants. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 133ddf3..d0772cd 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -10,6 +10,7 @@ This document records the behavior that the typed IR, LIR interpreter, and compi - Hindley-Milner inference generalizes top-level and local let bindings after applying substitutions. Recursive bindings are inferred monomorphically inside their recursive group and generalized afterward. - The active typed program stores top-level types but retains parser expressions in bodies. Compiler lowering therefore requires a new resolved, recursively typed result emitted by the inference pass; types cannot safely be reconstructed afterward. The legacy typed-program API projects this result's `TypedProgram`, so it does not maintain a second inference path. - Resolved inference exposes parser-independent semantic statement, expression, and pattern trees whose nodes carry their final instantiated type, span, lexical reference kind, resolved primitive operation, and named/structural record-coercion evidence. It also exposes deterministic symbols for every lexical binding category and links each captured reference to its target ID. Repeated occurrences of one pattern binding reuse one ID so equality constraints cannot be lost during lowering. Type-variable numbering in resolved schemes and reference types is normalized independently of prior inference calls. +- Interpolation placeholder expressions retain their absolute file, line, and column positions. Identical placeholder text in different lexical scopes therefore resolves to the parameter or pattern binding visible at its actual source location. - HIR reuses those resolved IDs unchanged for source symbols even when type and value declarations are interleaved; compiler-synthesized temporaries and constructor payload parameters occupy IDs above the resolved frontend range. - Top-level statements execute in source order. Every non-final expression must have type `unit`; the final expression is the program result. - LIR expression children execute strictly from left to right and stop at the first failure; later arguments, aggregate fields, interpolation parts, map entries, and operands are not evaluated after that failure. @@ -34,6 +35,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Maps currently accept string keys through inference and have deterministic ordered-map behavior. Runtime support for integer map keys is legacy representation detail, not permission for newly compiled programs. - Declared records retain declaration identity for named annotations. Structural record annotations retain field-shape compatibility and may accept additional fields where current inference permits it. Inference records coercion evidence at the accepting application; HIR and LIR retain an explicit conversion node so backends convert representations without changing compatibility rules. - HIR, LIR, and version-1 artifacts carry ordinary record and union declarations separately from nominal references. Generated Go uses concrete structs for ordinary named records, copies record updates, and performs explicit structural/named conversions; projection, patterns, formatting, and recursive equality remain shape-compatible. +- HIR, LIR, and version-1 artifacts also carry the intrinsic `Environment` record and `FsKind` union. Structural record conversions include width conversions from a record with extra fields to a required structural or nominal shape, and recursive named representations are validated without unbounded type expansion. - Generated Go represents each ordinary declared union as its own tagged struct with case-specific payload fields. Runtime union matching, formatting, and recursive equality consume a narrow conversion interface, preserving FScript behavior without erasing declarations to one storage type. - Discriminated unions retain their declared type name, case name, and optional payload. Recursive types require `type rec`; mutually recursive type-definition graphs are rejected. - `typeof` produces an FScript type token and `nameof` produces the resolved source name. diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index 24dc86f..d319ea7 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -20,8 +20,10 @@ module GoBackend = currentTypes.Value |> Option.bind (Map.tryFind name) let private declaredTypeName (declaration: Lir.TypeDeclaration) = $"record{Lir.symbolNumber declaration.Id}" let private tryRecordDeclaration name = - tryDeclaredType name - |> Option.bind (fun declaration -> match declaration.Import, declaration.Definition with None, TRecord fields -> Some(declaration, fields) | _ -> None) + if name = "Environment" then None + else + tryDeclaredType name + |> Option.bind (fun declaration -> match declaration.Import, declaration.Definition with None, TRecord fields -> Some(declaration, fields) | _ -> None) let private tryUnionDeclaration name = tryDeclaredType name |> Option.bind (fun declaration -> match declaration.Import, declaration.Definition with None, TUnion(_, cases) -> Some(declaration, cases) | _ -> None) diff --git a/src/FScript.Compiler/HirLowering.fs b/src/FScript.Compiler/HirLowering.fs index 4ba114a..21b1400 100644 --- a/src/FScript.Compiler/HirLowering.fs +++ b/src/FScript.Compiler/HirLowering.fs @@ -202,19 +202,41 @@ module HirLowering = match statement.Node with | TypeInfer.ResolvedTypeDeclaration definition -> Some(statement, definition) | _ -> None) + let declaredTypeNames = typeDefinitions |> List.map (snd >> _.Binding.Name) |> Set.ofList + let builtinTypeDefinitions = + [ "Environment", TRecord(Map.ofList [ "ScriptName", TOption TString; "Arguments", TList TString ]) + "FsKind", TUnion("FsKind", Map.ofList [ "File", Some TString; "Directory", Some TString; "Missing", None ]) ] + |> List.filter (fun (name, _) -> not (Set.contains name declaredTypeNames)) + |> List.map (fun (name, definition) -> name, definition, findSymbol state TypeInfer.TypeSymbol name syntheticSpan) let unionDefinitions = typeDefinitions |> List.choose (fun (statement, definition) -> match definition.Definition with | TUnion(name, cases) -> Some(statement, definition, name, cases) | _ -> None) - let constructorSymbols = - unionDefinitions - |> List.collect (fun (_, definition, name, cases) -> + let constructorDefinitions = + let referencedSymbols = resolved.SymbolReferences |> List.map _.SymbolId |> Set.ofList + let builtins = + builtinTypeDefinitions + |> List.collect (fun (name, definition, _) -> + match definition with + | TUnion(_, cases) -> + cases + |> Map.toList + |> List.map (fun (caseName, payload) -> + name, caseName, payload, syntheticSpan, + findSymbol state TypeInfer.UnionCaseSymbol $"{name}.{caseName}" syntheticSpan) + |> List.filter (fun (_, _, _, _, id) -> Set.contains (Hir.symbolNumber id) referencedSymbols) + | _ -> []) + let source = + unionDefinitions + |> List.collect (fun (_, definition, name, cases) -> cases |> Map.toList - |> List.map (fun (caseName, _) -> - name, caseName, findSymbol state TypeInfer.UnionCaseSymbol $"{name}.{caseName}" definition.Binding.Span)) + |> List.map (fun (caseName, payload) -> + name, caseName, payload, definition.Binding.Span, + findSymbol state TypeInfer.UnionCaseSymbol $"{name}.{caseName}" definition.Binding.Span)) + builtins @ source let mutable declarations = let builtinDeclarations = BuiltinSignatures.builtinSchemes |> Map.toList |> List.sortBy fst |> List.map (fun (name, scheme) -> @@ -231,11 +253,8 @@ module HirLowering = | PortableRuntimeIntrinsic -> Hir.Intrinsic external.Name | UnsupportedHostExtern -> Hir.HostExternal(external.Name, external.Arity) ({ Id = externSymbols.[external.Name]; Name = external.Name; Scheme = schemeFor externSymbols.[external.Name] external.Scheme; Attributes = []; IsRecursive = false; IsExported = false; Export = None; Span = span; Value = { Type = valueType; Span = span; Node = node } }: Hir.Declaration)) - builtinDeclarations @ externalDeclarations @ (constructorSymbols |> List.map (fun (typeName, caseName, id) -> - let _, definition, _, cases = unionDefinitions |> List.find (fun (_, _, name, _) -> name = typeName) + builtinDeclarations @ externalDeclarations @ (constructorDefinitions |> List.map (fun (typeName, caseName, payloadType, span, id) -> let unionType = TNamed typeName - let payloadType = cases.[caseName] - let span = definition.Binding.Span let value = match payloadType with | None -> ({ Type = unionType; Span = span; Node = Hir.UnionCase(typeName, caseName, None) }: Hir.Expression) @@ -348,15 +367,24 @@ module HirLowering = Fields = definition.Fields Span = definition.Binding.Span }: Hir.ImportedRecord))) let hirTypes = - typeDefinitions - |> List.map (fun (statement, definition) -> + (builtinTypeDefinitions + |> List.map (fun (name, definition, id) -> + ({ Id = id + Name = name + IsRecursive = false + Definition = definition + Attributes = [] + Import = None + Span = syntheticSpan }: Hir.TypeDeclaration))) + @ (typeDefinitions + |> List.map (fun (statement, definition) -> ({ Id = Hir.SymbolId definition.Binding.SymbolId Name = definition.Binding.Name IsRecursive = definition.IsRecursive Definition = definition.Definition Attributes = attributesOf statement Import = importedRecords |> List.tryFind (fun imported -> imported.Name = definition.Binding.Name) - Span = definition.Binding.Span }: Hir.TypeDeclaration)) + Span = definition.Binding.Span }: Hir.TypeDeclaration))) let diagnostics = errors @ state.Diagnostics if diagnostics.IsEmpty then Ok { Types = hirTypes; ImportedRecords = importedRecords; Declarations = declarations; Entry = entry } else Error diagnostics diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index c9555cb..a80f0c6 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -131,4 +131,9 @@ module HirValidation = | values -> List.min values let invalid = unresolved |> Set.filter (fun id -> Hir.symbolNumber id >= firstDeclaration) if invalid.IsEmpty then Ok program - else Error [ Diagnostic.hir "FSH1102" "HIR contains unresolved symbol references." None ] + else + let identifiers = + invalid + |> Seq.map (Hir.symbolNumber >> string) + |> String.concat ", " + Error [ Diagnostic.hir "FSH1102" $"HIR contains unresolved symbol references: {identifiers}." None ] diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index aee9a01..2c82e8f 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -442,7 +442,25 @@ module LirValidation = let resolve = function | TNamed name as named -> typeDefinitions |> Map.tryFind name |> Option.defaultValue named | value -> value - let compatible left right = compatible (resolve left) (resolve right) + let compatible left right = + let rec loop visited left right = + let resolvedLeft, resolvedRight = resolve left, resolve right + let pair = resolvedLeft, resolvedRight + if Set.contains pair visited then true + else + let visited = Set.add pair visited + match resolvedLeft, resolvedRight with + | TVar _, _ | _, TVar _ -> true + | TFun(la, lr), TFun(ra, rr) -> loop visited la ra && loop visited lr rr + | TTuple left, TTuple right -> left.Length = right.Length && List.forall2 (loop visited) left right + | TList left, TList right | TOption left, TOption right | TTask left, TTask right -> loop visited left right + | TMap(lk, lv), TMap(rk, rv) -> loop visited lk rk && loop visited lv rv + | TRecord left, TRecord right -> + left.Count = right.Count + && (left |> Map.forall (fun name item -> right |> Map.tryFind name |> Option.exists (loop visited item))) + | TUnion(ln, _), TUnion(rn, _) -> ln = rn + | left, right -> left = right + loop Set.empty left right let functionIds = program.Functions |> List.map _.Id let functions = program.Functions |> List.map (fun fn -> fn.Id, fn) |> Map.ofList let invalidImports = @@ -477,7 +495,7 @@ module LirValidation = program.Declarations |> List.iter (fun declaration -> if declaration.IsExported <> declaration.Export.IsSome || (declaration.Export |> Option.exists (fun export -> export.Name <> declaration.Name || not (validSpan export.Span))) then report "FSL2015" "LIR export metadata is inconsistent." declaration.Span) - let rec projectionType current = function + let rec projectionType checks traversed current = function | [] -> Some current | step :: rest -> let next = @@ -487,14 +505,17 @@ module LirValidation = | Lir.TupleItem index, TTuple items when index >= 0 && index < items.Length -> Some items.[index] | Lir.RecordField name, TRecord fields -> Map.tryFind name fields | Lir.OptionValue, TOption item -> Some item - // The preceding union-case check selects the concrete - // payload. Binding/check types validate that selection. - | Lir.UnionPayload, TUnion _ -> Some(TVar -1) + | Lir.UnionPayload, TUnion(_, cases) -> + checks + |> List.tryPick (function + | Lir.UnionCaseCheck(projection, _, caseName, true) when projection = traversed -> + Map.tryFind caseName cases |> Option.flatten + | _ -> None) | _ -> None - next |> Option.bind (fun item -> projectionType item rest) + next |> Option.bind (fun item -> projectionType checks (traversed @ [ step ]) item rest) let rec checkPlan expected (plan: Lir.PatternPlan) = checkSpan plan.Span - let at projection = projectionType expected projection + let at projection = projectionType plan.Checks [] expected projection let require projection predicate message = match at projection with Some value when predicate (resolve value) -> () | _ -> report "FSL2003" message plan.Span for check in plan.Checks do @@ -579,7 +600,22 @@ module LirValidation = | Lir.Fail value -> check value | Lir.DecisionTree(target, cases) -> check target; cases |> List.iter (fun case -> let caseLocals = Set.union locals (patternBindings case.Plan) in checkPlan target.Type case.Plan; case.Guard |> Option.iter (fun guard -> checkExpression caseLocals guard; if not (compatible guard.Type TBool) then report "FSL2007" "LIR decision guard must be Boolean." guard.Span); checkExpression caseLocals case.Body; if not (compatible expression.Type case.Body.Type) then report "FSL2007" "LIR decision body type is inconsistent." case.Body.Span) | Lir.Range(first, last) -> check first; check last; if not (compatible first.Type TInt && compatible last.Type TInt && compatible expression.Type (TList TInt)) then report "FSL2004" "LIR range representation is inconsistent." expression.Span - | Lir.RecordUpdate(target, fields) -> check target; fields |> List.iter (snd >> check); if not (compatible expression.Type target.Type) then report "FSL2004" "LIR record update representation is inconsistent." expression.Span + | Lir.RecordUpdate(target, fields) -> + check target + fields |> List.iter (snd >> check) + match resolve target.Type, resolve expression.Type with + | TRecord targetFields, TRecord resultFields -> + let preservesTarget = + targetFields + |> Map.forall (fun name fieldType -> + resultFields |> Map.tryFind name |> Option.exists (compatible fieldType)) + let validUpdates = + fields + |> List.forall (fun (name, value) -> + resultFields |> Map.tryFind name |> Option.exists (compatible value.Type)) + if not (preservesTarget && validUpdates) then + report "FSL2004" "LIR record update representation is inconsistent." expression.Span + | _ -> report "FSL2004" "LIR record update representation is inconsistent." expression.Span | Lir.RecordConvert(target, targetType) -> check target; match resolve target.Type, resolve targetType with TRecord _, TRecord _ when compatible expression.Type targetType -> () | _ -> report "FSL2004" "LIR record conversion evidence is inconsistent." expression.Span | Lir.IndexGet(target, index) -> check target; check index; match target.Type, expression.Type with TList item, TOption result when compatible index.Type TInt && compatible item result -> () | TMap(key, value), TOption result when compatible key index.Type && compatible value result -> () | _ -> report "FSL2004" "LIR index representation is inconsistent." expression.Span | Lir.Interpolate parts -> parts |> List.iter (function Lir.TextPart _ -> () | Lir.ValuePart value -> check value); if not (compatible expression.Type TString) then report "FSL2004" "LIR interpolation must produce string." expression.Span diff --git a/src/FScript.Language/Lexer.fs b/src/FScript.Language/Lexer.fs index 69e5826..2295954 100644 --- a/src/FScript.Language/Lexer.fs +++ b/src/FScript.Language/Lexer.fs @@ -162,14 +162,14 @@ module Lexer = let raw = if contentLen <= 0 then "" else src.Substring(start, contentLen) raw, idx, cLine, cCol - let tokenizeWithSourceName (sourceName: string option) (src: string) : Token list = + let tokenizeWithSourceNameAndOrigin (sourceName: string option) (startLine: int) (startColumn: int) (src: string) : Token list = let tokens = ResizeArray() let indentStack = ResizeArray() indentStack.Add(0) let mutable i = 0 - let mutable line = 1 - let mutable col = 1 + let mutable line = startLine + let mutable col = startColumn let mutable atLineStart = true let mutable inLetBindingHead = false let mutable parenDepth = 0 @@ -412,5 +412,8 @@ module Lexer = addToken EOF (mkSpan sourceName line col 1) tokens tokens |> Seq.toList + let tokenizeWithSourceName (sourceName: string option) (src: string) : Token list = + tokenizeWithSourceNameAndOrigin sourceName 1 1 src + let tokenize (src: string) : Token list = tokenizeWithSourceName None src diff --git a/src/FScript.Language/Parser.fs b/src/FScript.Language/Parser.fs index 5d9fa13..7f9aecc 100644 --- a/src/FScript.Language/Parser.fs +++ b/src/FScript.Language/Parser.fs @@ -60,12 +60,12 @@ module Parser = | BoolLit v -> LBool v | _ -> raise (ParseException { Message = "Expected literal"; Span = t.Span }) - let rec parseProgramWithSourceName (sourceName: string option) (src: string) : Program = - let tokens = Lexer.tokenizeWithSourceName sourceName src + let rec private parseProgramWithSourceNameAndOrigin (sourceName: string option) (startLine: int) (startColumn: int) (src: string) : Program = + let tokens = Lexer.tokenizeWithSourceNameAndOrigin sourceName startLine startColumn src let stream = TokenStream(tokens) - let parseSingleExpression (exprText: string) = - let parsed = parseProgramWithSourceName None exprText + let parseSingleExpression (originLine: int) (originColumn: int) (exprText: string) = + let parsed = parseProgramWithSourceNameAndOrigin sourceName originLine originColumn exprText match parsed with | [ SExpr expr ] -> expr | _ -> @@ -110,7 +110,15 @@ module Parser = let exprText = s.Substring(start, i - start - 1) if System.String.IsNullOrWhiteSpace(exprText) then raise (ParseException { Message = "Interpolation placeholder cannot be empty"; Span = span }) - let expr = parseSingleExpression exprText + let mutable originLine = span.Start.Line + let mutable originColumn = span.Start.Column + 2 + for offset = 0 to start - 1 do + if s.[offset] = '\n' then + originLine <- originLine + 1 + originColumn <- 1 + else + originColumn <- originColumn + 1 + let expr = parseSingleExpression originLine originColumn exprText expr, i let mutable i = 0 @@ -1697,5 +1705,8 @@ module Parser = stream.SkipNewlines() program |> Seq.toList + let parseProgramWithSourceName (sourceName: string option) (src: string) : Program = + parseProgramWithSourceNameAndOrigin sourceName 1 1 src + let parseProgram (src: string) : Program = parseProgramWithSourceName None src diff --git a/src/FScript.Language/TypeInfer.fs b/src/FScript.Language/TypeInfer.fs index 96d97f7..3423644 100644 --- a/src/FScript.Language/TypeInfer.fs +++ b/src/FScript.Language/TypeInfer.fs @@ -1674,6 +1674,11 @@ module TypeInfer = globalEnvironment <- Map.add name (allocate name BuiltinSymbol syntheticSpan (Some scheme)) globalEnvironment for external in externs |> List.sortBy _.Name do globalEnvironment <- Map.add external.Name (allocate external.Name ExternSymbol syntheticSpan (Some external.Scheme)) globalEnvironment + for definition in BuiltinTypes.typeDefs do + globalEnvironment <- Map.add definition.Name (allocate definition.Name TypeSymbol definition.Span None) globalEnvironment + for caseName, _ in definition.Cases do + let id = allocate $"{definition.Name}.{caseName}" UnionCaseSymbol definition.Span None + globalEnvironment <- globalEnvironment |> Map.add caseName id |> Map.add $"{definition.Name}.{caseName}" id for statement in typed do match statement with | TSType definition -> @@ -1969,16 +1974,33 @@ module TypeInfer = | ResolvedForEach(_, source, body) -> [ source; body ] | ResolvedConstant _ | ResolvedReference _ | ResolvedNone | ResolvedTypeToken _ -> [] let coercion = - match expression with - | EApply(fn, argument, _) -> - match finalType fn, finalType argument with - | TFun((TNamed target as targetType), _), (TRecord _ as sourceType) -> Some { SourceType = sourceType; TargetType = targetType; TargetName = Some target } - | TFun((TRecord _ as targetType), _), (TNamed source as sourceType) -> Some { SourceType = sourceType; TargetType = targetType; TargetName = Some source } + let recordShape = function + | TRecord fields -> Some fields + | TNamed name -> + match Map.tryFind name resolvedTypeDefinitions with + | Some(TRecord fields) -> Some fields + | _ -> None + | _ -> None + let conversion targetType sourceType = + match recordShape targetType, recordShape sourceType with + | Some targetFields, Some sourceFields + when targetType <> sourceType + && targetFields + |> Map.forall (fun name targetField -> + sourceFields |> Map.tryFind name |> Option.exists ((=) targetField)) -> + Some + { SourceType = sourceType + TargetType = targetType + TargetName = match targetType with TNamed name -> Some name | _ -> None } + | _ -> None + match node with + | ResolvedApply(fn, argument) -> + match fn.Type with + | TFun(targetType, _) -> conversion targetType argument.Type | _ -> None - | EBinOp("|>", argument, fn, _) -> - match finalType fn, finalType argument with - | TFun((TNamed target as targetType), _), (TRecord _ as sourceType) -> Some { SourceType = sourceType; TargetType = targetType; TargetName = Some target } - | TFun((TRecord _ as targetType), _), (TNamed source as sourceType) -> Some { SourceType = sourceType; TargetType = targetType; TargetName = Some source } + | ResolvedPipe(argument, fn) -> + match fn.Type with + | TFun(targetType, _) -> conversion targetType argument.Type | _ -> None | _ -> None { Type = expressionType; Span = Ast.spanOfExpr expression; Reference = reference diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index 52bf189..c6edb33 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -52,13 +52,13 @@ type HirTests() = member _.``Canonical HIR add snapshot is stable`` () = let compiled = compiledAdd () let declarations = compiled.Hir.Declarations |> List.filter (fun declaration -> declaration.Name = "add" || declaration.Name.StartsWith("$expression")) - Assert.That(Hir.format { compiled.Hir with Declarations = declarations }, Is.EqualTo(snapshot "add.hir")) + Assert.That(Hir.format { compiled.Hir with Types = []; Declarations = declarations }, Is.EqualTo(snapshot "add.hir")) [] member _.``Canonical LIR add snapshot is stable`` () = let compiled = compiledAdd () let declarations = compiled.Lir.Declarations |> List.filter (fun declaration -> declaration.Name = "add" || declaration.Name.StartsWith("$expression")) - Assert.That(Lir.format { compiled.Lir with Declarations = declarations }, Is.EqualTo(snapshot "add.lir")) + Assert.That(Lir.format { compiled.Lir with Types = []; Declarations = declarations }, Is.EqualTo(snapshot "add.lir")) [] member _.``HIR and LIR preserve named type declarations`` () = @@ -67,8 +67,8 @@ type HirTests() = match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with | Ok value -> value | Error diagnostics -> failwith (System.String.Join("; ", diagnostics |> List.map _.Message)) - let hirType = compiled.Hir.Types |> List.exactlyOne - let lirType = compiled.Lir.Types |> List.exactlyOne + let hirType = compiled.Hir.Types |> List.find (fun item -> item.Name = "Person") + let lirType = compiled.Lir.Types |> List.find (fun item -> item.Name = "Person") Assert.That(hirType.Name, Is.EqualTo("Person")) Assert.That((match hirType.Definition with TRecord _ -> true | _ -> false), Is.True) Assert.That(lirType.Name, Is.EqualTo("Person")) @@ -89,7 +89,8 @@ type HirTests() = Assert.That(declarationId "first", Is.EqualTo(resolvedId TypeInfer.DeclarationSymbol "first")) Assert.That(declarationId "Choice.Pick", Is.EqualTo(resolvedId TypeInfer.UnionCaseSymbol "Choice.Pick")) Assert.That(declarationId "apply", Is.EqualTo(resolvedId TypeInfer.DeclarationSymbol "apply")) - Assert.That(compiled.Hir.Types.Head.Id |> Hir.symbolNumber, Is.EqualTo(resolvedId TypeInfer.TypeSymbol "Choice")) + let choiceType = compiled.Hir.Types |> List.find (fun item -> item.Name = "Choice") + Assert.That(choiceType.Id |> Hir.symbolNumber, Is.EqualTo(resolvedId TypeInfer.TypeSymbol "Choice")) let rec bindings (expression: Hir.Expression) = let patternBindings (pattern: Hir.Pattern) = let rec collect (pattern: Hir.Pattern) = @@ -138,7 +139,8 @@ type HirTests() = member _.``HIR attaches structured export and import attributes`` () = let source = "[] type Person = { Name: string }\n[] let answer = 42" let compiled = match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with Ok value -> value | Error diagnostics -> failwith (System.String.Join("; ", diagnostics |> List.map _.Message)) - Assert.That((compiled.Hir.Types.Head.Attributes |> List.map _.Name) = [ "Import" ], Is.True) + let personType = compiled.Hir.Types |> List.find (fun item -> item.Name = "Person") + Assert.That((personType.Attributes |> List.map _.Name) = [ "Import" ], Is.True) let answer = compiled.Hir.Declarations |> List.find (fun declaration -> declaration.Name = "answer") Assert.That((answer.Attributes |> List.map _.Name) = [ "Export" ], Is.True) @@ -146,8 +148,9 @@ type HirTests() = member _.``HIR retains custom attributes on ordinary declarations`` () = let source = "[]\ntype Person = { Name: string }\n[] let answer = 42" let compiled = match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with Ok value -> value | Error diagnostics -> failwith (System.String.Join("; ", diagnostics |> List.map _.Message)) - Assert.That(compiled.Hir.Types.Head.Attributes.Head.Name, Is.EqualTo("BackendHint")) - Assert.That(compiled.Hir.Types.Head.Attributes.Head.Arguments = [ "fast"; "stable" ], Is.True) + let personType = compiled.Hir.Types |> List.find (fun item -> item.Name = "Person") + Assert.That(personType.Attributes.Head.Name, Is.EqualTo("BackendHint")) + Assert.That(personType.Attributes.Head.Arguments = [ "fast"; "stable" ], Is.True) let answer = compiled.Hir.Declarations |> List.find (fun declaration -> declaration.Name = "answer") Assert.That(answer.Attributes.Head.Name, Is.EqualTo("Trace")) @@ -202,3 +205,24 @@ type HirTests() = [ [ Lir.OptionValue; Lir.TupleItem 0 ] [ Lir.OptionValue; Lir.TupleItem 1 ] ] Assert.That((projections = expected), Is.True) + + [] + member _.``Compiler emits intrinsic types and validates their union payload plans`` () = + let source = "match Fs.kind \".\" with | FsKind.Directory path -> path | _ -> \"missing\"" + let compiled = + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Ok value -> value + | Error diagnostics -> failwith (System.String.Join("; ", diagnostics |> List.map _.Message)) + Assert.That(compiled.Lir.Types |> List.map _.Name, Does.Contain("Environment")) + Assert.That(compiled.Lir.Types |> List.map _.Name, Does.Contain("FsKind")) + let fsKind = compiled.Lir.Types |> List.find (fun item -> item.Name = "FsKind") + match fsKind.Definition with + | TUnion(_, cases) -> Assert.That(cases |> Map.containsKey "Directory", Is.True) + | _ -> Assert.Fail("Expected intrinsic FsKind union") + + [] + member _.``LIR validates recursive named representations without recursion overflow`` () = + let source = "type rec Tree = | Leaf of int | Branch of (Tree * Tree)\nlet rec sum tree = match tree with | Leaf value -> value | Branch (left, right) -> sum left + sum right\nsum (Branch (Leaf 1, Leaf 2))" + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Ok compiled -> Assert.That(compiled.Lir.Types |> List.map _.Name, Does.Contain("Tree")) + | Error diagnostics -> Assert.Fail(System.String.Join("; ", diagnostics |> List.map _.Message)) diff --git a/tests/FScript.Compiler.Tests/Snapshots/add.hir b/tests/FScript.Compiler.Tests/Snapshots/add.hir index d4d3176..1329533 100644 --- a/tests/FScript.Compiler.Tests/Snapshots/add.hir +++ b/tests/FScript.Compiler.Tests/Snapshots/add.hir @@ -1,14 +1,14 @@ -decl %72 add: ('a -> ('a -> int)) - lambda %73 x: 'a : ('a -> ('a -> int)) - lambda %74 y: 'a : ('a -> int) +decl %77 add: ('a -> ('a -> int)) + lambda %78 x: 'a : ('a -> ('a -> int)) + lambda %79 y: 'a : ('a -> int) primitive IntAdd : int - ref %73 : 'a - ref %74 : 'a -decl %75 $expression75: int + ref %78 : 'a + ref %79 : 'a +decl %80 $expression80: int apply : int apply : (int -> int) - ref %72 : (int -> (int -> int)) + ref %77 : (int -> (int -> int)) const int 1 : int const int 2 : int entry - ref %75 : int + ref %80 : int diff --git a/tests/FScript.Compiler.Tests/Snapshots/add.lir b/tests/FScript.Compiler.Tests/Snapshots/add.lir index f097ae0..f0c11e0 100644 --- a/tests/FScript.Compiler.Tests/Snapshots/add.lir +++ b/tests/FScript.Compiler.Tests/Snapshots/add.lir @@ -1,16 +1,16 @@ -function @0 %73 x captures=[]: 'a -> ('a -> int) - make-closure @1 captures=[73] : ('a -> int) -function @1 %74 y captures=[73]: 'a -> int +function @0 %78 x captures=[]: 'a -> ('a -> int) + make-closure @1 captures=[78] : ('a -> int) +function @1 %79 y captures=[78]: 'a -> int operation IntAdd : int - load %73 : 'a - load %74 : 'a -decl %72 add: ('a -> ('a -> int)) + load %78 : 'a + load %79 : 'a +decl %77 add: ('a -> ('a -> int)) make-closure @0 captures=[] : ('a -> ('a -> int)) -decl %75 $expression75: int +decl %80 $expression80: int call-closure : int call-closure : (int -> int) - load %72 : (int -> (int -> int)) + load %77 : (int -> (int -> int)) const Int 1L : int const Int 2L : int entry - load %75 : int + load %80 : int diff --git a/tests/FScript.Language.Tests/ParserTests.fs b/tests/FScript.Language.Tests/ParserTests.fs index 2cde0aa..c77c1af 100644 --- a/tests/FScript.Language.Tests/ParserTests.fs +++ b/tests/FScript.Language.Tests/ParserTests.fs @@ -719,6 +719,17 @@ type ParserTests () = | SExpr (EInterpolatedString ([ IPText "hello "; IPExpr (EVar ("name", _)) ], _)) -> () | _ -> Assert.Fail("Expected interpolated string") + [] + member _.``Interpolation placeholders retain absolute source spans`` () = + let program = Parser.parseProgramWithSourceName (Some "sample.fss") "let name = \"Ada\"\n$\"hello {name}\"" + match program with + | [ _; SExpr(EInterpolatedString([ IPText "hello "; IPExpr(EVar("name", span)) ], _)) ] -> + Assert.That(span.Start.File, Is.EqualTo(Some "sample.fss")) + Assert.That(span.Start.Line, Is.EqualTo(2)) + Assert.That(span.Start.Column, Is.EqualTo(10)) + Assert.That(span.End.Column, Is.EqualTo(14)) + | _ -> Assert.Fail("Expected interpolated variable placeholder") + [] member _.``Parses interpolated placeholder containing string literal`` () = let p = Helpers.parse "$\"{if true then \"missing\" else \"none\"}\"" diff --git a/tests/FScript.Language.Tests/ResolvedInferenceTests.fs b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs index 7ff9dc9..3c39e3a 100644 --- a/tests/FScript.Language.Tests/ResolvedInferenceTests.fs +++ b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs @@ -99,6 +99,42 @@ type ResolvedInferenceTests() = match evidence.SourceType with TRecord fields -> Assert.That(fields.Count, Is.EqualTo(2)) | _ -> Assert.Fail("Expected structural record source") | None -> Assert.Fail("Expected structural-to-named record coercion evidence") + [] + member _.``Resolved interpolation references retain their lexical parameter`` () = + let source = "let first (value: int) = $\"{value}\"\nlet second (value: string) = $\"{value}\"\nsecond \"ok\"" + let resolved = Helpers.parse source |> TypeInfer.inferResolvedProgram + let parameterIds = + resolved.Symbols + |> List.filter (fun symbol -> symbol.Kind = TypeInfer.ParameterSymbol && symbol.Name = "value") + |> List.map _.SymbolId + let interpolationReferences = + resolved.ResolvedStatements + |> List.collect _.Expressions + |> List.collect flatten + |> List.choose (fun expression -> + match expression.Node with + | TypeInfer.ResolvedReference reference when reference.Name = "value" -> Some(reference.SymbolId, reference.Type) + | _ -> None) + Assert.That(parameterIds.Length, Is.EqualTo(2)) + let expected = [ (parameterIds.[0], TInt); (parameterIds.[1], TString) ] + Assert.That((interpolationReferences = expected), Is.True) + + [] + member _.``Resolved applications retain structural width coercion evidence`` () = + let source = "let city (value: {| City: string |}) = value.City\ncity {| City = \"Paris\"; Zip = 75000 |}" + let resolved = Helpers.parse source |> TypeInfer.inferResolvedProgram + let evidence = + resolved.ResolvedStatements + |> List.collect _.Expressions + |> List.collect flatten + |> List.choose _.RecordCoercion + |> List.exactlyOne + match evidence.SourceType, evidence.TargetType with + | TRecord sourceFields, TRecord targetFields -> + Assert.That(sourceFields.Count, Is.EqualTo(2)) + Assert.That(targetFields.Count, Is.EqualTo(1)) + | _ -> Assert.Fail("Expected structural record width coercion") + [] member _.``Resolved statements preserve structured export metadata`` () = let resolved = Helpers.parse "[] let answer = 42" |> TypeInfer.inferResolvedProgram From d95ab32cdf7449e84eb7b25d1a366e9154d8fab8 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 22:13:21 +0200 Subject: [PATCH 061/133] fix(compiler): harden IR validation invariants --- CHANGELOG.md | 2 + docs/architecture/typed-ir-and-backends.md | 2 +- docs/specs/compiler-semantics-baseline.md | 2 +- src/FScript.Compiler/HirValidation.fs | 73 +++++++++++++++++-- src/FScript.Compiler/Lir.fs | 55 ++++++++++---- tests/FScript.Compiler.Tests/HirTests.fs | 29 ++++++++ .../FScript.Compiler.Tests/IrArtifactTests.fs | 17 +++++ 7 files changed, 154 insertions(+), 26 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c2af80e..cfdc462 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Rejected malformed HIR, LIR, and loaded IR artifacts containing unknown semantic types, nested invalid spans, or any unresolved symbol ID. + - Compiled every shipped sample by preserving interpolation scopes, intrinsic types, recursive representations, and structural record-width conversions. - Lowered source patterns into backend-neutral executable LIR projection and decision plans. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index dc46b12..bb7ba22 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -36,7 +36,7 @@ Compiler artifacts are closed over the language's intrinsic `Environment` record Resolved statements expose normalized structured attributes as name, argument list, and source span. The parser retains unknown attributes instead of assigning semantics to them, and HIR attaches them to ordinary value and type declarations while retaining dedicated export and target-import linkage for consumers that need semantic access without reparsing attribute arguments. -HIR validation rejects duplicate and unresolved symbols, invalid declaration spans, and schemes inconsistent with their values. LIR validation rechecks artifact-local references and spans, then verifies closure calls, branches and decision guards, primitive signatures, aggregate representations, declaration value types, intrinsic catalog membership, and complete target-import linkage. Artifact deserialization, LIR execution, and Go generation run the same validator, so malformed or directly constructed LIR cannot bypass compiler invariants. +HIR validation rejects duplicate or unresolved symbols (including low-numbered IDs), unknown named types, invalid normalized schemes, invalid nested spans and fields, and schemes inconsistent with their values. LIR validation rechecks every semantic type and nested span in addition to artifact-local references, then verifies closure calls, branches and decision guards, primitive signatures, aggregate representations, declaration value types, intrinsic catalog membership, and complete target-import linkage. Artifact deserialization, LIR execution, and Go generation run the same validator, so malformed or directly constructed LIR cannot bypass compiler invariants. Lexical `let` is retained as a structured LIR node. Its symbol is visible only in the body, or in both the binding and body for recursive functions. The interpreter closes the recursive environment explicitly; Go emission declares the function slot before assigning its closure. `raise` becomes an explicit `Fail` node so consumers preserve evaluation order and attach their own stage-specific failure diagnostic. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index d0772cd..55ed9e1 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -91,4 +91,4 @@ All entries are type-checkable and retained as linkage in HIR/LIR. Except for th Parse, type, and evaluation failures currently use source spans and exceptions. Compiler entry points translate expected failures into structured stage diagnostics while leaving existing embedding APIs behavior-compatible. -Version-1 `.fsir` artifacts require compiler and language versions, source metadata, ordered dependencies, and a complete program. Unknown formats and versions are rejected before LIR construction; malformed metadata and invalid reconstructed LIR receive `FSA5xxx` diagnostics. +Version-1 `.fsir` artifacts require compiler and language versions, source metadata, ordered dependencies, and a complete program. Unknown formats and versions are rejected before LIR construction; malformed metadata receives `FSA5xxx`, while reconstructed programs containing unknown named types, unresolved symbols, or invalid nested spans are rejected by the shared `FSL2xxx` validator before execution or generation. diff --git a/src/FScript.Compiler/HirValidation.fs b/src/FScript.Compiler/HirValidation.fs index a80f0c6..a35766e 100644 --- a/src/FScript.Compiler/HirValidation.fs +++ b/src/FScript.Compiler/HirValidation.fs @@ -43,12 +43,75 @@ module HirValidation = let invalidImports = program.ImportedRecords |> List.exists (fun record -> System.String.IsNullOrWhiteSpace(record.ModulePath) || System.String.IsNullOrWhiteSpace(record.MemberName)) || (program.ImportedRecords |> List.map _.Name |> List.distinct |> List.length) <> program.ImportedRecords.Length + let knownTypes = program.Types |> List.map _.Name |> Set.ofList + let rec validType (quantified: Set) = function + | TVar variable -> variable >= 0 && (quantified.IsEmpty || Set.contains variable quantified) + | TFun(argument, result) -> validType quantified argument && validType quantified result + | TTuple items -> items |> List.forall (validType quantified) + | TList item | TOption item | TTask item -> validType quantified item + | TMap(key, value) -> validType quantified key && validType quantified value + | TRecord fields -> fields |> Map.forall (fun name fieldType -> not (System.String.IsNullOrWhiteSpace name) && validType quantified fieldType) + | TUnion(name, cases) -> + Set.contains name knownTypes + && cases |> Map.forall (fun caseName payload -> not (System.String.IsNullOrWhiteSpace caseName) && payload |> Option.forall (validType quantified)) + | TNamed name -> Set.contains name knownTypes + | _ -> true + let validClosedType = validType Set.empty + let validScheme (Forall(variables, valueType)) = + variables = [ 0 .. variables.Length - 1 ] && validType (Set.ofList variables) valueType + let rec validPattern (pattern: Hir.Pattern) = + validSpan pattern.Span + && validClosedType pattern.Type + && match pattern.Node with + | Hir.ConsPattern(head, tail) -> validPattern head && validPattern tail + | Hir.TuplePattern items -> items |> List.forall validPattern + | Hir.RecordPattern fields -> fields |> List.forall (fun (name, value) -> not (System.String.IsNullOrWhiteSpace name) && validPattern value) + | Hir.MapPattern(clauses, tail) -> clauses |> List.forall (fun (key, value) -> validPattern key && validPattern value) && tail |> Option.forall validPattern + | Hir.SomePattern item -> validPattern item + | Hir.UnionPattern(typeName, caseName, payload) -> Set.contains typeName knownTypes && not (System.String.IsNullOrWhiteSpace caseName) && payload |> Option.forall validPattern + | Hir.TypePattern patternType -> validClosedType patternType + | _ -> true + let rec validExpression (expression: Hir.Expression) = + let valid = validExpression + validSpan expression.Span + && validClosedType expression.Type + && match expression.Node with + | Hir.Lambda(_, name, parameterType, body) -> not (System.String.IsNullOrWhiteSpace name) && validClosedType parameterType && valid body + | Hir.Apply(fn, argument) | Hir.Pipe(argument, fn) | Hir.Cons(fn, argument) | Hir.Append(fn, argument) + | Hir.Range(fn, argument) | Hir.IndexGet(fn, argument) -> valid fn && valid argument + | Hir.If(condition, whenTrue, whenFalse) -> valid condition && valid whenTrue && valid whenFalse + | Hir.Primitive(_, arguments) | Hir.Tuple arguments | Hir.List arguments -> arguments |> List.forall valid + | Hir.Record fields -> fields |> List.forall (fun (name, value) -> not (System.String.IsNullOrWhiteSpace name) && valid value) + | Hir.FieldGet(target, name) -> valid target && not (System.String.IsNullOrWhiteSpace name) + | Hir.SomeValue target | Hir.Fail target -> valid target + | Hir.Let(_, name, _, binding, body) -> not (System.String.IsNullOrWhiteSpace name) && valid binding && valid body + | Hir.LetRecursiveGroup(bindings, body) -> + bindings |> List.forall (fun (_, name, bindingType, value) -> not (System.String.IsNullOrWhiteSpace name) && validClosedType bindingType && valid value) + && valid body + | Hir.Match(target, cases) -> + valid target + && cases |> List.forall (fun case -> validSpan case.Span && validPattern case.Pattern && case.Guard |> Option.forall valid && valid case.Body) + | Hir.RecordUpdate(target, fields) -> valid target && fields |> List.forall (fun (name, value) -> not (System.String.IsNullOrWhiteSpace name) && valid value) + | Hir.RecordConvert(target, targetType) -> valid target && validClosedType targetType + | Hir.Interpolate parts -> parts |> List.forall (function Hir.TextPart _ -> true | Hir.ValuePart value -> valid value) + | Hir.UnionCase(typeName, caseName, payload) -> Set.contains typeName knownTypes && not (System.String.IsNullOrWhiteSpace caseName) && payload |> Option.forall valid + | Hir.Map entries -> entries |> List.forall (function Hir.KeyValue(key, value) -> valid key && valid value | Hir.Spread value -> valid value) + | Hir.ForEach(_, name, itemType, source, body) -> not (System.String.IsNullOrWhiteSpace name) && validClosedType itemType && valid source && valid body + | Hir.TypeToken tokenType -> validClosedType tokenType + | _ -> true + let invalidTypeReferences = + program.Types |> List.exists (fun declaration -> not (validClosedType declaration.Definition)) + || program.ImportedRecords |> List.exists (fun record -> record.Fields |> List.exists (snd >> validClosedType >> not)) + || program.Declarations |> List.exists (fun declaration -> not (validScheme declaration.Scheme) || not (validExpression declaration.Value)) + || program.Entry |> Option.exists (validExpression >> not) if invalidAttributes then Error [ Diagnostic.hir "FSH1108" "HIR structured attribute metadata is invalid or inconsistent." None ] elif invalidTypes then Error [ Diagnostic.hir "FSH1107" "HIR type declarations must be unique, well-formed, and consistently linked." None ] elif invalidImports then Error [ Diagnostic.hir "FSH1105" "HIR imported record linkage must be complete and uniquely named." None ] + elif invalidTypeReferences then + Error [ Diagnostic.hir "FSH1109" "HIR contains an unresolved or invalid semantic type, nested span, or field name." None ] elif not duplicateIds.IsEmpty then Error [ Diagnostic.hir "FSH1101" "HIR declaration IDs must be unique." None ] elif program.Declarations |> List.exists (fun declaration -> declaration.IsExported <> declaration.Export.IsSome || (declaration.Export |> Option.exists (fun export -> export.Name <> declaration.Name || not (validSpan export.Span)))) then @@ -124,16 +187,10 @@ module HirValidation = | None -> () ] |> Set.ofList |> Set.filter (fun id -> not (Set.contains id declared)) - // IDs below the first declaration are reserved deterministic builtin symbols. - let firstDeclaration = - match declarationIds |> List.map Hir.symbolNumber with - | [] -> 0 - | values -> List.min values - let invalid = unresolved |> Set.filter (fun id -> Hir.symbolNumber id >= firstDeclaration) - if invalid.IsEmpty then Ok program + if unresolved.IsEmpty then Ok program else let identifiers = - invalid + unresolved |> Seq.map (Hir.symbolNumber >> string) |> String.concat ", " Error [ Diagnostic.hir "FSH1102" $"HIR contains unresolved symbol references: {identifiers}." None ] diff --git a/src/FScript.Compiler/Lir.fs b/src/FScript.Compiler/Lir.fs index 2c82e8f..5461b4d 100644 --- a/src/FScript.Compiler/Lir.fs +++ b/src/FScript.Compiler/Lir.fs @@ -439,6 +439,7 @@ module LirValidation = let ids = program.Declarations |> List.map _.Id let typeIds = program.Types |> List.map _.Id let typeDefinitions = program.Types |> List.map (fun declaration -> declaration.Name, declaration.Definition) |> Map.ofList + let knownTypes = typeDefinitions |> Map.keys |> Set.ofSeq let resolve = function | TNamed name as named -> typeDefinitions |> Map.tryFind name |> Option.defaultValue named | value -> value @@ -461,19 +462,40 @@ module LirValidation = | TUnion(ln, _), TUnion(rn, _) -> ln = rn | left, right -> left = right loop Set.empty left right + let validSpan (span: Span) = + span.Start.Line >= 0 && span.Start.Column >= 0 && span.End.Line >= 0 && span.End.Column >= 0 + && (span.Start.File <> span.End.File + || span.End.Line > span.Start.Line + || (span.End.Line = span.Start.Line && span.End.Column >= span.Start.Column)) + let rec validType = function + | TVar variable -> variable >= 0 + | TFun(argument, result) -> validType argument && validType result + | TTuple items -> items |> List.forall validType + | TList item | TOption item | TTask item -> validType item + | TMap(key, value) -> validType key && validType value + | TRecord fields -> fields |> Map.forall (fun name fieldType -> not (System.String.IsNullOrWhiteSpace name) && validType fieldType) + | TUnion(name, cases) -> + Set.contains name knownTypes + && cases |> Map.forall (fun caseName payload -> not (System.String.IsNullOrWhiteSpace caseName) && payload |> Option.forall validType) + | TNamed name -> Set.contains name knownTypes + | _ -> true let functionIds = program.Functions |> List.map _.Id let functions = program.Functions |> List.map (fun fn -> fn.Id, fn) |> Map.ofList let invalidImports = - program.ImportedRecords |> List.exists (fun record -> System.String.IsNullOrWhiteSpace(record.ModulePath) || System.String.IsNullOrWhiteSpace(record.MemberName)) + program.ImportedRecords |> List.exists (fun record -> + System.String.IsNullOrWhiteSpace(record.ModulePath) || System.String.IsNullOrWhiteSpace(record.MemberName) + || not (validSpan record.Span) + || record.Fields |> List.exists (fun (name, fieldType) -> System.String.IsNullOrWhiteSpace name || not (validType fieldType))) || (program.ImportedRecords |> List.map _.Name |> List.distinct |> List.length) <> program.ImportedRecords.Length let invalidTypes = (program.Types |> List.map _.Name |> List.distinct |> List.length) <> program.Types.Length || (typeIds @ ids |> List.distinct |> List.length) <> typeIds.Length + ids.Length || program.Types |> List.exists (fun declaration -> - match declaration.Definition with - | TRecord _ -> false - | TUnion(name, _) -> name <> declaration.Name - | _ -> true) + not (validSpan declaration.Span) || not (validType declaration.Definition) + || match declaration.Definition with + | TRecord _ -> false + | TUnion(name, _) -> name <> declaration.Name + | _ -> true) if invalidTypes then Error [ { Code = "FSL2016"; Stage = Lowering; Message = "LIR type declarations must have unique IDs and names with valid definitions."; Span = None } ] elif invalidImports then @@ -486,12 +508,8 @@ module LirValidation = let diagnostics = ResizeArray() let report code message span = diagnostics.Add({ Code = code; Stage = Lowering; Message = message; Span = Some span }) - let validSpan (span: Span) = - span.Start.Line >= 0 && span.Start.Column >= 0 && span.End.Line >= 0 && span.End.Column >= 0 - && (span.Start.File <> span.End.File - || span.End.Line > span.Start.Line - || (span.End.Line = span.Start.Line && span.End.Column >= span.Start.Column)) let checkSpan span = if not (validSpan span) then report "FSL2002" "LIR source span is invalid." span + let checkType span valueType = if not (validType valueType) then report "FSL2017" "LIR contains an unresolved or invalid semantic type." span program.Declarations |> List.iter (fun declaration -> if declaration.IsExported <> declaration.Export.IsSome || (declaration.Export |> Option.exists (fun export -> export.Name <> declaration.Name || not (validSpan export.Span))) then report "FSL2015" "LIR export metadata is inconsistent." declaration.Span) @@ -515,6 +533,7 @@ module LirValidation = next |> Option.bind (fun item -> projectionType checks (traversed @ [ step ]) item rest) let rec checkPlan expected (plan: Lir.PatternPlan) = checkSpan plan.Span + checkType plan.Span expected let at projection = projectionType plan.Checks [] expected projection let require projection predicate message = match at projection with Some value when predicate (resolve value) -> () | _ -> report "FSL2003" message plan.Span @@ -527,8 +546,9 @@ module LirValidation = | Lir.SomeCheck projection | Lir.NoneCheck projection -> require projection (function TOption _ -> true | _ -> false) "LIR option plan requires an option projection." | Lir.UnionCaseCheck(projection, typeName, caseName, hasPayload) -> require projection (function TUnion(actual, cases) when actual = typeName -> Map.tryFind caseName cases |> Option.exists (fun payload -> payload.IsSome = hasPayload) | _ -> false) "LIR union plan has inconsistent case metadata." - | Lir.TypeCheck(projection, _) -> require projection (fun _ -> true) "LIR type-test plan has an invalid projection." + | Lir.TypeCheck(projection, patternType) -> checkType plan.Span patternType; require projection (fun _ -> true) "LIR type-test plan has an invalid projection." for binding in plan.Bindings do + checkType plan.Span binding.Type require binding.Projection (compatible binding.Type) "LIR pattern binding has an inconsistent projection type." for mapPlan in plan.Maps do match at mapPlan.Projection |> Option.map resolve with @@ -547,6 +567,7 @@ module LirValidation = let declared = ids |> Set.ofList let rec checkExpression locals (expression: Lir.Expression) = checkSpan expression.Span + checkType expression.Span expression.Type let check = checkExpression locals match expression.Node with | Lir.Constant value when not (compatible expression.Type (literalType value)) -> report "FSL2004" "LIR constant has an inconsistent type." expression.Span @@ -596,7 +617,7 @@ module LirValidation = | Lir.Cons(head, tail) -> check head; check tail; match expression.Type, tail.Type with TList item, TList tailItem when compatible item head.Type && compatible item tailItem -> () | _ -> report "FSL2004" "LIR cons representation is inconsistent." expression.Span | Lir.Append(left, right) -> check left; check right; if not (compatible expression.Type left.Type && compatible expression.Type right.Type) then report "FSL2004" "LIR append representation is inconsistent." expression.Span | Lir.Let(id, _, recursive, binding, body) -> let scope = Set.add id locals in checkExpression (if recursive then scope else locals) binding; checkExpression scope body; if not (compatible expression.Type body.Type) then report "FSL2004" "LIR let result type is inconsistent." expression.Span - | Lir.LetRecursiveGroup(bindings, body) -> let scope = bindings |> List.fold (fun value (id, _, _, _) -> Set.add id value) locals in bindings |> List.iter (fun (_, _, bindingType, value) -> checkExpression scope value; if not (compatible bindingType value.Type) then report "FSL2006" "LIR recursive binding signature is inconsistent." value.Span); checkExpression scope body; if not (compatible expression.Type body.Type) then report "FSL2004" "LIR recursive group result is inconsistent." expression.Span + | Lir.LetRecursiveGroup(bindings, body) -> let scope = bindings |> List.fold (fun value (id, _, _, _) -> Set.add id value) locals in bindings |> List.iter (fun (_, _, bindingType, value) -> checkType value.Span bindingType; checkExpression scope value; if not (compatible bindingType value.Type) then report "FSL2006" "LIR recursive binding signature is inconsistent." value.Span); checkExpression scope body; if not (compatible expression.Type body.Type) then report "FSL2004" "LIR recursive group result is inconsistent." expression.Span | Lir.Fail value -> check value | Lir.DecisionTree(target, cases) -> check target; cases |> List.iter (fun case -> let caseLocals = Set.union locals (patternBindings case.Plan) in checkPlan target.Type case.Plan; case.Guard |> Option.iter (fun guard -> checkExpression caseLocals guard; if not (compatible guard.Type TBool) then report "FSL2007" "LIR decision guard must be Boolean." guard.Span); checkExpression caseLocals case.Body; if not (compatible expression.Type case.Body.Type) then report "FSL2007" "LIR decision body type is inconsistent." case.Body.Span) | Lir.Range(first, last) -> check first; check last; if not (compatible first.Type TInt && compatible last.Type TInt && compatible expression.Type (TList TInt)) then report "FSL2004" "LIR range representation is inconsistent." expression.Span @@ -616,7 +637,7 @@ module LirValidation = if not (preservesTarget && validUpdates) then report "FSL2004" "LIR record update representation is inconsistent." expression.Span | _ -> report "FSL2004" "LIR record update representation is inconsistent." expression.Span - | Lir.RecordConvert(target, targetType) -> check target; match resolve target.Type, resolve targetType with TRecord _, TRecord _ when compatible expression.Type targetType -> () | _ -> report "FSL2004" "LIR record conversion evidence is inconsistent." expression.Span + | Lir.RecordConvert(target, targetType) -> check target; checkType expression.Span targetType; match resolve target.Type, resolve targetType with TRecord _, TRecord _ when compatible expression.Type targetType -> () | _ -> report "FSL2004" "LIR record conversion evidence is inconsistent." expression.Span | Lir.IndexGet(target, index) -> check target; check index; match target.Type, expression.Type with TList item, TOption result when compatible index.Type TInt && compatible item result -> () | TMap(key, value), TOption result when compatible key index.Type && compatible value result -> () | _ -> report "FSL2004" "LIR index representation is inconsistent." expression.Span | Lir.Interpolate parts -> parts |> List.iter (function Lir.TextPart _ -> () | Lir.ValuePart value -> check value); if not (compatible expression.Type TString) then report "FSL2004" "LIR interpolation must produce string." expression.Span | Lir.UnionCase(typeName, caseName, payload) -> payload |> Option.iter check; match resolve expression.Type with TUnion(actual, cases) when actual = typeName && Map.containsKey caseName cases -> () | _ -> report "FSL2004" "LIR union representation is inconsistent." expression.Span @@ -624,16 +645,18 @@ module LirValidation = | Lir.Intrinsic name when not (Map.containsKey name BuiltinSignatures.builtinSchemes || Map.containsKey name RuntimeIntrinsics.schemes) -> report "FSL2009" $"Unknown LIR intrinsic '{name}'." expression.Span | Lir.HostExternal(_, arity) when arity < 0 -> report "FSL2010" "LIR host external arity cannot be negative." expression.Span | Lir.TargetImport(modulePath, memberName) when System.String.IsNullOrWhiteSpace(modulePath) || System.String.IsNullOrWhiteSpace(memberName) -> report "FSL2011" "LIR target import linkage is incomplete." expression.Span - | Lir.ForEach(id, _, itemType, source, body) -> check source; checkExpression (Set.add id locals) body; match source.Type with TList actual when compatible actual itemType && compatible expression.Type TUnit -> () | _ -> report "FSL2004" "LIR for-loop representation is inconsistent." expression.Span - | Lir.TypeToken _ when not (compatible expression.Type TTypeToken) -> report "FSL2004" "LIR type token has an inconsistent type." expression.Span + | Lir.ForEach(id, _, itemType, source, body) -> checkType expression.Span itemType; check source; checkExpression (Set.add id locals) body; match source.Type with TList actual when compatible actual itemType && compatible expression.Type TUnit -> () | _ -> report "FSL2004" "LIR for-loop representation is inconsistent." expression.Span + | Lir.TypeToken tokenType -> checkType expression.Span tokenType; if not (compatible expression.Type TTypeToken) then report "FSL2004" "LIR type token has an inconsistent type." expression.Span | _ -> () program.Functions |> List.iter (fun fn -> checkSpan fn.Span + checkType fn.Span fn.ParameterType + checkType fn.Span fn.ReturnType let expected = LirLowering.capturedReferences (Set.singleton fn.Parameter) fn.Body |> Set.toList |> List.sortBy Lir.symbolNumber if fn.Captures <> expected then report "FSL2013" "Lifted LIR function captures do not match its ordered free-symbol set." fn.Span checkExpression (Set.add fn.Parameter (Set.ofList fn.Captures)) fn.Body if not (compatible fn.ReturnType fn.Body.Type) then report "FSL2006" "Lifted LIR function return type is inconsistent." fn.Span) - program.Declarations |> List.iter (fun declaration -> checkSpan declaration.Span; checkExpression Set.empty declaration.Value; if not (compatible declaration.Type declaration.Value.Type) then report "FSL2004" $"LIR declaration '{declaration.Name}' type is inconsistent." declaration.Span) + program.Declarations |> List.iter (fun declaration -> checkSpan declaration.Span; checkType declaration.Span declaration.Type; checkExpression Set.empty declaration.Value; if not (compatible declaration.Type declaration.Value.Type) then report "FSL2004" $"LIR declaration '{declaration.Name}' type is inconsistent." declaration.Span) program.Entry |> Option.iter (checkExpression Set.empty) if diagnostics.Count = 0 then Ok program else Error(List.ofSeq diagnostics) diff --git a/tests/FScript.Compiler.Tests/HirTests.fs b/tests/FScript.Compiler.Tests/HirTests.fs index c6edb33..c7154ec 100644 --- a/tests/FScript.Compiler.Tests/HirTests.fs +++ b/tests/FScript.Compiler.Tests/HirTests.fs @@ -164,6 +164,21 @@ type HirTests() = | Ok _ -> Assert.Fail("Expected invalid HIR diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSH1104")) + [] + member _.``HIR validation rejects unknown semantic types and every unresolved symbol ID`` () = + let unknown = ({ Type = TNamed "Missing"; Span = span; Node = Hir.NoneValue }: Hir.Expression) + match HirValidation.validate { Types = []; ImportedRecords = []; Declarations = []; Entry = Some unknown } with + | Ok _ -> Assert.Fail("Expected unknown HIR type diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSH1109")) + + let reference = ({ Type = TUnit; Span = span; Node = Hir.Reference(Hir.SymbolId 0) }: Hir.Expression) + let declaration: Hir.Declaration = + { Id = Hir.SymbolId 10; Name = "value"; Scheme = Forall([], TUnit) + Attributes = []; IsRecursive = false; IsExported = false; Export = None; Span = span; Value = reference } + match HirValidation.validate { Types = []; ImportedRecords = []; Declarations = [ declaration ]; Entry = None } with + | Ok _ -> Assert.Fail("Expected unresolved HIR symbol diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSH1102")) + [] member _.``LIR validation rejects inconsistent control flow`` () = let constant valueType value = ({ Type = valueType; Span = span; Node = Lir.Constant value }: Lir.Expression) @@ -181,6 +196,20 @@ type HirTests() = | Ok _ -> Assert.Fail("Expected unavailable primitive diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSL2009")) + [] + member _.``LIR validation rejects unknown types and invalid nested spans`` () = + let unknown = ({ Type = TNamed "Missing"; Span = span; Node = Lir.NoneValue }: Lir.Expression) + match LirValidation.validate { Types = []; ImportedRecords = []; Functions = []; Declarations = []; Entry = Some unknown } with + | Ok _ -> Assert.Fail("Expected unknown LIR type diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSL2017")) + + let invalidSpan = Span.mk (Span.pos -1 0) (Span.pos -1 1) + let child = ({ Type = TInt; Span = invalidSpan; Node = Lir.Constant(Lir.Int 1L) }: Lir.Expression) + let tuple = ({ Type = TTuple [ TInt ]; Span = span; Node = Lir.Tuple [ child ] }: Lir.Expression) + match LirValidation.validate { Types = []; ImportedRecords = []; Functions = []; Declarations = []; Entry = Some tuple } with + | Ok _ -> Assert.Fail("Expected nested LIR span diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSL2002")) + [] member _.``Initial LIR optimization pass is identity`` () = let expression = ({ Type = TInt; Span = span; Node = Lir.Constant(Lir.Int 1L) }: Lir.Expression) diff --git a/tests/FScript.Compiler.Tests/IrArtifactTests.fs b/tests/FScript.Compiler.Tests/IrArtifactTests.fs index 96bd804..d89de8b 100644 --- a/tests/FScript.Compiler.Tests/IrArtifactTests.fs +++ b/tests/FScript.Compiler.Tests/IrArtifactTests.fs @@ -77,6 +77,23 @@ type IrArtifactTests() = | Ok _ -> Assert.Fail("Expected invalid version-metadata diagnostic") | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSA5005")) + [] + member _.``IR artifact post-load validation rejects unknown semantic types`` () = + let artifact = compile "1" |> _.Lir |> IrArtifact.serialize |> parseObject + let required (name: string) (value: JsonObject) : JsonNode = + match value[name] with + | null -> failwith $"Expected '{name}'" + | node -> node + let program = (required "program" artifact).AsObject() + let entry = (required "entry" program).AsObject() + let unknownType = JsonObject() + unknownType["kind"] <- "named" + unknownType["name"] <- "Missing" + entry["type"] <- unknownType + match IrArtifact.deserialize (Encoding.UTF8.GetBytes(artifact.ToJsonString())) with + | Ok _ -> Assert.Fail("Expected post-load semantic-type diagnostic") + | Error diagnostics -> Assert.That(diagnostics |> List.map _.Code, Does.Contain("FSL2017")) + [] member _.``IR artifact records normalized source paths and SHA256 hashes`` () = let path = IO.Path.Combine(IO.Path.GetTempPath(), $"fscript-source-{Guid.NewGuid():N}.fss") From 34fac2528cc17b86d9855e1fabe593493e81ff13 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 22:18:46 +0200 Subject: [PATCH 062/133] test(go): complete structural failure conformance --- CHANGELOG.md | 2 + docs/architecture/typed-ir-and-backends.md | 2 +- docs/specs/compiler-semantics-baseline.md | 1 + .../GoBackendTests.fs | 19 ++++++++ .../ConformanceTests.fs | 45 ++++++++++++++----- 5 files changed, 58 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index cfdc462..416f52b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Enforced deterministic non-generic Go output and cross-runtime conformance for structural width, recursive records, and distinct failure categories. + - Rejected malformed HIR, LIR, and loaded IR artifacts containing unknown semantic types, nested invalid spans, or any unresolved symbol ID. - Compiled every shipped sample by preserving interpolation scopes, intrinsic types, recursive representations, and structural record-width conversions. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index bb7ba22..79ba8b3 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -96,7 +96,7 @@ Imported Go record types use `[] type Name = ## Cross-runtime conformance -The conformance suite evaluates one source program through the legacy evaluator, a serialized-and-deserialized LIR program, and a generated Go executable. It compares successful results and failure categories across nested structural equality, named/structural record conversion, unions, higher-order and polymorphic functions, mutual recursion, and ordered maps. Named semantic types are resolved to their record or union representation while HIR is built, so a named record crossing a function boundary cannot accidentally receive the union representation in a downstream consumer. +The conformance suite evaluates one source program through the legacy evaluator, a serialized-and-deserialized LIR program, and a generated Go executable. It compares successful results across nested structural equality, named/structural width conversion, unions, higher-order and polymorphic functions, mutual recursion, recursive records, and ordered maps; it also normalizes raised errors, exhausted matches, and division by zero into shared failure categories. Named semantic types are resolved to their record or union representation while HIR is built, so a named record crossing a function boundary cannot accidentally receive the union representation in a downstream consumer. ## Previous experiment diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 55ed9e1..fc94830 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -50,6 +50,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Compiled map patterns consume entries deterministically in map order, merge repeated bindings only when structurally equal, and bind an optional tail to the remaining ordered map. Compiled type-reference patterns use resolved types rather than source text lookup. - Record updates evaluate the base once and then updates in source order. Ranges, loops, interpolation, indexing, raise, append, and cons preserve the evaluator's existing evaluation order and failures. - Equality is structural for primitives, tuples, records, maps, options, lists, unions, and type tokens, recursively. Functions, tasks, constructors, and extern handles are not structurally equal. Backend host-language comparability must not leak into this behavior. +- Cross-runtime failure conformance classifies explicit `raise`, exhausted pattern matches, and integer division by zero independently of each host runtime's exception or panic text. - Cross-runtime tests compare the legacy evaluator with artifact-round-tripped LIR execution and generated Go for successful values and failure categories. Named records passed through typed function boundaries keep record field projection semantics; their nominal annotation must not select a union runtime representation. - Cross-runtime result formatting quotes strings at every nesting level, renders unit as `()`, renders ordered maps as `map { key => value }`, uses compact semicolon separators for lists, and retains comma-space separators for tuples. - HIR and LIR are accepted only when artifact-local references resolve, source spans are valid, declaration and expression types agree, calls and control-flow nodes have consistent signatures, aggregate representations match their semantic types, and referenced primitives/imports are available and complete. diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index a31c702..45fce66 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -3,6 +3,7 @@ namespace FScript.Backend.Go.Tests open System open System.Diagnostics open System.IO +open System.Text.RegularExpressions open FScript.Compiler open FScript.Language open NUnit.Framework @@ -124,6 +125,24 @@ type GoBackendTests() = Assert.That(program, Does.Contain("Case0 int64")) Assert.That(program, Does.Contain("FScriptUnion() fsruntime.Union")) + [] + member _.``Go generation is deterministic targets Go 1.25 and emits no generic declarations`` () = + let compiled = + match Compiler.compile { Program = Parser.parseProgram "let identity value = value\nidentity [1; 2]"; Externs = [] } with + | Ok value -> value + | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + let generate () = + match GoBackend.generate { ModuleName = "fscript.generated/deterministic" } compiled.Lir with + | Ok files -> files + | Error diagnostics -> failwith (String.Join("; ", diagnostics |> List.map _.Message)) + let first = generate () + let second = generate () + Assert.That((first = second), Is.True) + let goMod = first |> List.find (fun file -> file.RelativePath = "go.mod") |> _.Content + Assert.That(goMod, Does.Contain("go 1.25")) + for file in first |> List.filter (fun file -> file.RelativePath.EndsWith(".go")) do + Assert.That(Regex.IsMatch(file.Content, @"func\s+(?:\([^)]*\)\s*)?[A-Za-z_][A-Za-z0-9_]*\s*\["), Is.False, file.RelativePath) + [] member _.``Go emitter delegates LIR pattern plans to the runtime`` () = let source = "match Some (1, 2) with | Some (left, right) -> left + right | None -> 0" diff --git a/tests/FScript.Conformance.Tests/ConformanceTests.fs b/tests/FScript.Conformance.Tests/ConformanceTests.fs index bbf920b..ffce423 100644 --- a/tests/FScript.Conformance.Tests/ConformanceTests.fs +++ b/tests/FScript.Conformance.Tests/ConformanceTests.fs @@ -9,6 +9,7 @@ open NUnit.Framework module private Harness = type Outcome = { Value: string; Stdout: string list } + type FailureCategory = Raised | MatchFailure | DivisionByZero let private quoteString (value: string) = "\"" + value.Replace("\\", "\\\\").Replace("\"", "\\\"").Replace("\r", "\\r").Replace("\n", "\\n").Replace("\t", "\\t") + "\"" @@ -82,6 +83,30 @@ module private Harness = finally if Directory.Exists directory then Directory.Delete(directory, true) + let legacyFailure source = + try + legacy source |> ignore + None + with + | :? DivideByZeroException -> Some DivisionByZero + | EvalException error when error.Message = "No match cases matched" -> Some MatchFailure + | EvalException _ -> Some Raised + + let lirFailure source = + match lir source with + | Ok _ -> None + | Error "FSI3005" -> Some DivisionByZero + | Error "FSI3012" -> Some MatchFailure + | Error "FSI3011" -> Some Raised + | Error code -> failwith $"Unexpected LIR failure category {code}" + + let goFailure source = + match go source with + | Ok _ -> None + | Error error when error.Contains("integer divide by zero") || error.Contains("division by zero") -> Some DivisionByZero + | Error error when error.Contains("Match failure.") -> Some MatchFailure + | Error _ -> Some Raised + [] type ConformanceTests() = [] @@ -122,6 +147,8 @@ type ConformanceTests() = [ state + value) 0 (Map.ofList [(\"a\", 1); (\"b\", 2)]), Map.filter (fun key value -> value > 1) (Map.ofList [(\"a\", 1); (\"b\", 2)]), Map.choose (fun key value -> if value > 1 then Some value else None) (Map.ofList [(\"a\", 1); (\"b\", 2)]), Map.map (fun value -> value + 1) (Map.ofList [(\"a\", 1)]), Map.iter (fun key value -> ignore value) (Map.ofList [(\"a\", 1)]), Map.remove \"a\" (Map.ofList [(\"a\", 1); (\"b\", 2)]))")>] [] [ String.concat \",\"")>] + [] + [ node.Value + sum next | None -> node.Value\nsum { Value = 1; Next = Some { Value = 2; Next = None } }")>] member _.``Legacy LIR artifact and Go results conform`` (source: string) = let expected = Harness.legacy source match Harness.lir source with @@ -135,13 +162,11 @@ type ConformanceTests() = Assert.That(System.String.Join("\n", actual.Stdout), Is.EqualTo(System.String.Join("\n", expected.Stdout)), source) | Error error -> Assert.Fail($"Go failed with {error}: {source}") - [] - member _.``Failure categories conform`` () = - let source = "raise \"boom\"" - Assert.Throws(fun () -> Harness.legacy source |> ignore) |> ignore - match Harness.lir source with - | Error code -> Assert.That(code, Is.EqualTo("FSI3011")) - | Ok value -> Assert.Fail($"Expected LIR failure, got {value}") - match Harness.go source with - | Ok value -> Assert.Fail($"Expected generated Go failure, got {value}") - | Error error -> Assert.That(error, Does.Contain("boom")) + [] + [ 0")>] + [] + member _.``Failure categories conform`` (source: string) = + let expected = Harness.legacyFailure source + Assert.That(expected.IsSome, Is.True, source) + Assert.That(Harness.lirFailure source, Is.EqualTo(expected), source) + Assert.That(Harness.goFailure source, Is.EqualTo(expected), source) From dcbd84b374113295322ec2aac35b2219476df013 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 22:24:15 +0200 Subject: [PATCH 063/133] fix(language): preserve top-level shadowing identities --- CHANGELOG.md | 2 + docs/architecture/typed-ir-and-backends.md | 2 + docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Language/TypeInfer.fs | 52 ++++++++++++++++--- .../ConformanceTests.fs | 1 + .../ResolvedInferenceTests.fs | 14 +++++ 6 files changed, 65 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 416f52b..ab5ae88 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Preserved sequential top-level shadowing identities through resolved inference, LIR interpretation, and generated Go. + - Enforced deterministic non-generic Go output and cross-runtime conformance for structural width, recursive records, and distinct failure categories. - Rejected malformed HIR, LIR, and loaded IR artifacts containing unknown semantic types, nested invalid spans, or any unresolved symbol ID. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index 79ba8b3..cd72043 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -14,6 +14,8 @@ source -> parser/includes -> inference + resolved typed tree -> HIR -> LIR Resolved inference assigns deterministic program-local IDs to builtins, externs, types, union cases, declarations, parameters, pattern bindings, and locals in normalized source traversal. Declaration schemes and instantiated reference types normalize type variables from zero, so results do not depend on the process-global fresh-variable counter. The legacy `TypedProgram` remains a projection with unchanged public entry points. +Top-level reference resolution walks declarations in source order. A non-recursive initializer sees the preceding environment, a recursive declaration or group sees its own complete group, and a later shadowing declaration replaces the name only after its initializer is resolved. Reference occurrences are consumed in traversal order as well as matched by name and span, so repeated included source locations cannot collapse distinct lexical identities. + `FScript.Compiler` is a reusable, initially non-packaged .NET assembly. It owns HIR, HIR validation, lowering, LIR, LIR validation, and artifact serialization. It depends on `FScript.Language`; the language assembly never depends on the compiler. Validated LIR consumers are separate leaf assemblies. `FScript.Interpreter.Lir` owns execution and runtime LIR values. `FScript.Backend.Go` owns representation selection, Go emission, and embedded runtime templates. Both reference `FScript.Compiler`, while the compiler references neither consumer. Public modules retain the `FScript.Compiler` namespace so the API surface remains source-compatible while assembly ownership is explicit. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index fc94830..1a7c0dc 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -11,6 +11,7 @@ This document records the behavior that the typed IR, LIR interpreter, and compi - The active typed program stores top-level types but retains parser expressions in bodies. Compiler lowering therefore requires a new resolved, recursively typed result emitted by the inference pass; types cannot safely be reconstructed afterward. The legacy typed-program API projects this result's `TypedProgram`, so it does not maintain a second inference path. - Resolved inference exposes parser-independent semantic statement, expression, and pattern trees whose nodes carry their final instantiated type, span, lexical reference kind, resolved primitive operation, and named/structural record-coercion evidence. It also exposes deterministic symbols for every lexical binding category and links each captured reference to its target ID. Repeated occurrences of one pattern binding reuse one ID so equality constraints cannot be lost during lowering. Type-variable numbering in resolved schemes and reference types is normalized independently of prior inference calls. - Interpolation placeholder expressions retain their absolute file, line, and column positions. Identical placeholder text in different lexical scopes therefore resolves to the parameter or pattern binding visible at its actual source location. +- Top-level declarations resolve sequentially: a non-recursive shadowing initializer refers to the preceding binding, recursive groups see every peer atomically, and subsequent expressions refer to the newest binding. - HIR reuses those resolved IDs unchanged for source symbols even when type and value declarations are interleaved; compiler-synthesized temporaries and constructor payload parameters occupy IDs above the resolved frontend range. - Top-level statements execute in source order. Every non-final expression must have type `unit`; the final expression is the program result. - LIR expression children execute strictly from left to right and stop at the first failure; later arguments, aggregate fields, interpolation parts, map entries, and operands are not evaluated after that failure. diff --git a/src/FScript.Language/TypeInfer.fs b/src/FScript.Language/TypeInfer.fs index 3423644..0a383d0 100644 --- a/src/FScript.Language/TypeInfer.fs +++ b/src/FScript.Language/TypeInfer.fs @@ -1679,6 +1679,7 @@ module TypeInfer = for caseName, _ in definition.Cases do let id = allocate $"{definition.Name}.{caseName}" UnionCaseSymbol definition.Span None globalEnvironment <- globalEnvironment |> Map.add caseName id |> Map.add $"{definition.Name}.{caseName}" id + let builtinEnvironment = globalEnvironment for statement in typed do match statement with | TSType definition -> @@ -1776,15 +1777,39 @@ module TypeInfer = | ERange(first, last, _) -> visitSame first; visitSame last | EInterpolatedString(parts, _) -> parts |> List.iter (function IPText _ -> () | IPExpr item -> visitSame item) | EUnit _ | ELiteral _ | ENone _ -> () + let allocated kind name span = + symbols + |> Seq.find (fun symbol -> symbol.Kind = kind && symbol.Name = name && symbol.Span = span) + |> _.SymbolId + let mutable resolutionEnvironment = builtinEnvironment for statement in typed do match statement with - | TSLet(_, expression, _, _, _, _) -> visit globalEnvironment expression - | TSLetPattern(pattern, expression, _, _, _) -> visit globalEnvironment expression; bindPattern PatternBindingSymbol globalEnvironment pattern |> ignore - | TSLetRecGroup(bindings, _, _) -> bindings |> List.iter (fun (_, expression, _, _) -> visit globalEnvironment expression) - | TSExpr expression -> visit globalEnvironment expression.Expr - | TSBackendImport _ -> () - | TSType _ -> () + | TSType definition -> + resolutionEnvironment <- Map.add definition.Name (allocated TypeSymbol definition.Name definition.Span) resolutionEnvironment + for caseName, _ in definition.Cases do + let id = allocated UnionCaseSymbol $"{definition.Name}.{caseName}" definition.Span + resolutionEnvironment <- resolutionEnvironment |> Map.add caseName id |> Map.add $"{definition.Name}.{caseName}" id + | TSLet(name, expression, _, recursive, _, span) -> + let id = allocated DeclarationSymbol name span + visit (if recursive then Map.add name id resolutionEnvironment else resolutionEnvironment) expression + resolutionEnvironment <- Map.add name id resolutionEnvironment + | TSLetPattern(pattern, expression, bindings, _, span) -> + visit resolutionEnvironment expression + bindPattern PatternBindingSymbol resolutionEnvironment pattern |> ignore + for name, _ in bindings |> Map.toList do + resolutionEnvironment <- Map.add name (allocated DeclarationSymbol name span) resolutionEnvironment + | TSLetRecGroup(bindings, _, _) -> + let recursiveEnvironment = + bindings + |> List.fold (fun environment (name, _, _, span) -> + Map.add name (allocated DeclarationSymbol name span) environment) resolutionEnvironment + bindings |> List.iter (fun (_, expression, _, _) -> visit recursiveEnvironment expression) + resolutionEnvironment <- recursiveEnvironment + | TSExpr expression -> visit resolutionEnvironment expression.Expr + | TSBackendImport(name, _, _, span) -> + resolutionEnvironment <- Map.add name (allocated DeclarationSymbol name span) resolutionEnvironment let resolvedReferences = references |> Seq.toList + let mutable remainingReferenceOccurrences = resolvedReferences let finalType expression = expressionTypes |> Map.tryFind expression |> Option.defaultValue TUnit |> normalizeTypeVariables let declarations = (BuiltinTypes.typeDefs |> List.map (fun definition -> definition.Name, definition)) @@ -1823,7 +1848,20 @@ module TypeInfer = { SymbolId = id; Name = name; Type = bindingType; Span = span } let referenceAt expression = let span = Ast.spanOfExpr expression - resolvedReferences |> List.tryFind (fun item -> item.Span = span) + let expectedName = + match expression with + | EVar(name, _) | ENameOf(name, _) | ETypeOf(name, _) -> Some name + | EFieldGet(EVar(moduleName, _), fieldName, _) -> Some $"{moduleName}.{fieldName}" + | _ -> None + let matches (item: ResolvedSymbolReference) = + item.Span = span && expectedName |> Option.forall ((=) item.Name) + let rec consume preceding = function + | [] -> None + | item :: tail when matches item -> + remainingReferenceOccurrences <- List.rev preceding @ tail + Some item + | item :: tail -> consume (item :: preceding) tail + consume [] remainingReferenceOccurrences let rec resolvePattern (expected: Type) (pattern: Pattern) : ResolvedPattern = let span = Ast.spanOfPattern pattern let make node : ResolvedPattern = { Type = expected; Span = span; Node = node } diff --git a/tests/FScript.Conformance.Tests/ConformanceTests.fs b/tests/FScript.Conformance.Tests/ConformanceTests.fs index ffce423..b431eef 100644 --- a/tests/FScript.Conformance.Tests/ConformanceTests.fs +++ b/tests/FScript.Conformance.Tests/ConformanceTests.fs @@ -149,6 +149,7 @@ type ConformanceTests() = [ String.concat \",\"")>] [] [ node.Value + sum next | None -> node.Value\nsum { Value = 1; Next = Some { Value = 2; Next = None } }")>] + [] member _.``Legacy LIR artifact and Go results conform`` (source: string) = let expected = Harness.legacy source match Harness.lir source with diff --git a/tests/FScript.Language.Tests/ResolvedInferenceTests.fs b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs index 3c39e3a..29e9ce4 100644 --- a/tests/FScript.Language.Tests/ResolvedInferenceTests.fs +++ b/tests/FScript.Language.Tests/ResolvedInferenceTests.fs @@ -119,6 +119,20 @@ type ResolvedInferenceTests() = let expected = [ (parameterIds.[0], TInt); (parameterIds.[1], TString) ] Assert.That((interpolationReferences = expected), Is.True) + [] + member _.``Resolved top-level shadowing keeps sequential lexical identities`` () = + let resolved = Helpers.parse "let value = 1\nlet value = value + 1\nvalue" |> TypeInfer.inferResolvedProgram + let declarations = + resolved.Symbols + |> List.filter (fun symbol -> symbol.Kind = TypeInfer.DeclarationSymbol && symbol.Name = "value") + |> List.map _.SymbolId + let references = + resolved.SymbolReferences + |> List.filter (fun reference -> reference.Kind = TypeInfer.DeclarationSymbol && reference.Name = "value") + |> List.map _.SymbolId + Assert.That(declarations.Length, Is.EqualTo(2)) + Assert.That((references = declarations), Is.True) + [] member _.``Resolved applications retain structural width coercion evidence`` () = let source = "let city (value: {| City: string |}) = value.City\ncity {| City = \"Paris\"; Zip = 75000 |}" From af81c4fbc7d6b4e23897a8637dbe885ff520c6ff Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 22:32:37 +0200 Subject: [PATCH 064/133] feat(language): short circuit boolean operators --- CHANGELOG.md | 2 ++ docs/specs/compiler-semantics-baseline.md | 3 ++- src/FScript.Interpreter.Lir/LirInterpreter.fs | 16 ++++++++++++ src/FScript.Language/Eval.fs | 25 +++++++++++++------ .../ConformanceTests.fs | 2 ++ .../LirInterpreterTests.fs | 10 ++++++++ tests/FScript.Language.Tests/EvalTests.fs | 5 ++++ 7 files changed, 54 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ab5ae88..c059b6c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added short-circuit evaluation for Boolean `&&` and `||` across interpreted and generated programs. + - Preserved sequential top-level shadowing identities through resolved inference, LIR interpretation, and generated Go. - Enforced deterministic non-generic Go output and cross-runtime conformance for structural width, recursive records, and distinct failure categories. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 1a7c0dc..7befcba 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -14,7 +14,8 @@ This document records the behavior that the typed IR, LIR interpreter, and compi - Top-level declarations resolve sequentially: a non-recursive shadowing initializer refers to the preceding binding, recursive groups see every peer atomically, and subsequent expressions refer to the newest binding. - HIR reuses those resolved IDs unchanged for source symbols even when type and value declarations are interleaved; compiler-synthesized temporaries and constructor payload parameters occupy IDs above the resolved frontend range. - Top-level statements execute in source order. Every non-final expression must have type `unit`; the final expression is the program result. -- LIR expression children execute strictly from left to right and stop at the first failure; later arguments, aggregate fields, interpolation parts, map entries, and operands are not evaluated after that failure. +- LIR expression children execute strictly from left to right and stop at the first failure; later arguments, aggregate fields, interpolation parts, map entries, and ordinary operands are not evaluated after that failure. Boolean operators additionally skip an unnecessary right operand. +- Boolean `&&` and `||` evaluate the left operand first and skip the right operand when the result is already known. The legacy evaluator, LIR interpreter, and generated Go share this short-circuit behavior. - Pipeline application retains its distinct value-then-function evaluation order through resolved inference, HIR, LIR, interpretation, serialization, and Go generation; it is not rewritten into ordinary function-first application. - Compiled top-level expressions execute exactly once in source order even when interleaved with declarations. Top-level pattern bindings evaluate their scrutinee once, fail before later statements when unmatched, and expose all bound names to subsequent statements. diff --git a/src/FScript.Interpreter.Lir/LirInterpreter.fs b/src/FScript.Interpreter.Lir/LirInterpreter.fs index 4a3f140..6456bb6 100644 --- a/src/FScript.Interpreter.Lir/LirInterpreter.fs +++ b/src/FScript.Interpreter.Lir/LirInterpreter.fs @@ -303,6 +303,22 @@ module LirInterpreter = | Ok(VBool false) -> evaluate environment whenFalse | Ok _ -> fail "FSI3004" expression.Span "Branch condition is not Boolean." | Error errors -> Error errors + | Lir.Operation(Lir.BooleanAnd, [ left; right ]) -> + evaluate environment left + |> Result.bind (function + | VBool false -> Ok(VBool false) + | VBool true -> + evaluate environment right + |> Result.bind (function VBool value -> Ok(VBool value) | _ -> fail "FSI3006" expression.Span "Primitive operands do not match their LIR operation.") + | _ -> fail "FSI3006" expression.Span "Primitive operands do not match their LIR operation.") + | Lir.Operation(Lir.BooleanOr, [ left; right ]) -> + evaluate environment left + |> Result.bind (function + | VBool true -> Ok(VBool true) + | VBool false -> + evaluate environment right + |> Result.bind (function VBool value -> Ok(VBool value) | _ -> fail "FSI3006" expression.Span "Primitive operands do not match their LIR operation.") + | _ -> fail "FSI3006" expression.Span "Primitive operands do not match their LIR operation.") | Lir.Operation(op, _) -> binary (fun left right -> match op, left, right with diff --git a/src/FScript.Language/Eval.fs b/src/FScript.Language/Eval.fs index f96937b..9d73a09 100644 --- a/src/FScript.Language/Eval.fs +++ b/src/FScript.Language/Eval.fs @@ -653,6 +653,22 @@ module Eval = match av, bv with | VList xs, VList ys -> VList (xs @ ys) | _ -> raise (EvalException { Message = "Both sides of '@' must be lists"; Span = span }) + | EBinOp ("&&", a, b, span) -> + match evalExpr runtime typeDefs env a with + | VBool false -> VBool false + | VBool true -> + match evalExpr runtime typeDefs env b with + | VBool value -> VBool value + | _ -> raise (EvalException { Message = "Boolean operands required"; Span = span }) + | _ -> raise (EvalException { Message = "Boolean operands required"; Span = span }) + | EBinOp ("||", a, b, span) -> + match evalExpr runtime typeDefs env a with + | VBool true -> VBool true + | VBool false -> + match evalExpr runtime typeDefs env b with + | VBool value -> VBool value + | _ -> raise (EvalException { Message = "Boolean operands required"; Span = span }) + | _ -> raise (EvalException { Message = "Boolean operands required"; Span = span }) | EBinOp (op, a, b, span) -> let av = evalExpr runtime typeDefs env a let bv = evalExpr runtime typeDefs env b @@ -662,8 +678,7 @@ module Eval = | VFloat x, VFloat y -> VFloat (fFloat x y) | _ -> raise (EvalException { Message = "Numeric operands required"; Span = span }) match op with - | "|>" -> - applyFunctionValue runtime evalExpr typeDefs span bv av + | "|>" -> applyFunctionValue runtime evalExpr typeDefs span bv av | "+" -> arith ( + ) ( + ) | "-" -> arith ( - ) ( - ) | "*" -> arith ( * ) ( * ) @@ -698,12 +713,6 @@ module Eval = | _ -> false VBool res | _ -> raise (EvalException { Message = "Numeric operands required"; Span = span }) - | "&&" | "||" -> - match av, bv with - | VBool x, VBool y -> - let res = if op = "&&" then x && y else x || y - VBool res - | _ -> raise (EvalException { Message = "Boolean operands required"; Span = span }) | "::" -> match bv with | VList xs -> VList (av :: xs) diff --git a/tests/FScript.Conformance.Tests/ConformanceTests.fs b/tests/FScript.Conformance.Tests/ConformanceTests.fs index b431eef..6df4bce 100644 --- a/tests/FScript.Conformance.Tests/ConformanceTests.fs +++ b/tests/FScript.Conformance.Tests/ConformanceTests.fs @@ -150,6 +150,8 @@ type ConformanceTests() = [] [ node.Value + sum next | None -> node.Value\nsum { Value = 1; Next = Some { Value = 2; Next = None } }")>] [] + [] + [] member _.``Legacy LIR artifact and Go results conform`` (source: string) = let expected = Harness.legacy source match Harness.lir source with diff --git a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs index c6d6237..8a15c41 100644 --- a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs +++ b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs @@ -34,6 +34,16 @@ type LirInterpreterTests() = | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VList [ LirValue.VInt 1L; LirValue.VInt 2L; LirValue.VInt 3L ])) | Error diagnostics -> Assert.Fail(errorText diagnostics) + [] + [] + member _.``Logical operations short circuit in LIR`` (source: string, expected: bool) = + match Compiler.compile { Program = Parser.parseProgram source; Externs = [] } with + | Error diagnostics -> Assert.Fail(errorText diagnostics) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VBool expected)) + | Error diagnostic -> Assert.Fail(errorText diagnostic) + [] [] [] diff --git a/tests/FScript.Language.Tests/EvalTests.fs b/tests/FScript.Language.Tests/EvalTests.fs index 7245292..441cae6 100644 --- a/tests/FScript.Language.Tests/EvalTests.fs +++ b/tests/FScript.Language.Tests/EvalTests.fs @@ -427,6 +427,11 @@ type EvalTests () = Helpers.eval "1 < 2" |> assertBool true Helpers.eval "true && false" |> assertBool false + [] + member _.``Logical operators short circuit their right operand`` () = + Helpers.eval "false && raise \"and-right\"" |> assertBool false + Helpers.eval "true || raise \"or-right\"" |> assertBool true + [] member _.``Evaluates pipeline operator`` () = Helpers.eval "let inc x = x + 1\n1 |> inc" |> assertInt 2L From 1abf29963da3000d5081139f8e4d29f61e6d04d1 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 22:33:36 +0200 Subject: [PATCH 065/133] test(compiler): cover runtime equality boundaries --- CHANGELOG.md | 2 ++ docs/architecture/typed-ir-and-backends.md | 2 +- tests/FScript.Conformance.Tests/ConformanceTests.fs | 3 +++ 3 files changed, 6 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c059b6c..3814399 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Enforced cross-runtime equality conformance for closures, type tokens, and unequal collections. + - Added short-circuit evaluation for Boolean `&&` and `||` across interpreted and generated programs. - Preserved sequential top-level shadowing identities through resolved inference, LIR interpretation, and generated Go. diff --git a/docs/architecture/typed-ir-and-backends.md b/docs/architecture/typed-ir-and-backends.md index cd72043..d87ecac 100644 --- a/docs/architecture/typed-ir-and-backends.md +++ b/docs/architecture/typed-ir-and-backends.md @@ -98,7 +98,7 @@ Imported Go record types use `[] type Name = ## Cross-runtime conformance -The conformance suite evaluates one source program through the legacy evaluator, a serialized-and-deserialized LIR program, and a generated Go executable. It compares successful results across nested structural equality, named/structural width conversion, unions, higher-order and polymorphic functions, mutual recursion, recursive records, and ordered maps; it also normalizes raised errors, exhausted matches, and division by zero into shared failure categories. Named semantic types are resolved to their record or union representation while HIR is built, so a named record crossing a function boundary cannot accidentally receive the union representation in a downstream consumer. +The conformance suite evaluates one source program through the legacy evaluator, a serialized-and-deserialized LIR program, and a generated Go executable. It compares successful results across nested structural equality, inequality, non-equal closures, type tokens, named/structural width conversion, unions, higher-order and polymorphic functions, mutual recursion, recursive records, and ordered maps; it also normalizes raised errors, exhausted matches, and division by zero into shared failure categories. Named semantic types are resolved to their record or union representation while HIR is built, so a named record crossing a function boundary cannot accidentally receive the union representation in a downstream consumer. ## Previous experiment diff --git a/tests/FScript.Conformance.Tests/ConformanceTests.fs b/tests/FScript.Conformance.Tests/ConformanceTests.fs index 6df4bce..bd49d14 100644 --- a/tests/FScript.Conformance.Tests/ConformanceTests.fs +++ b/tests/FScript.Conformance.Tests/ConformanceTests.fs @@ -152,6 +152,9 @@ type ConformanceTests() = [] [] [] + [ value) = (fun value -> value)")>] + [] + [] member _.``Legacy LIR artifact and Go results conform`` (source: string) = let expected = Harness.legacy source match Harness.lir source with From 1c788982a2b4e42532b63fac1d4681685542aaeb Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 22:37:45 +0200 Subject: [PATCH 066/133] fix(compiler): complete numeric and list failures --- CHANGELOG.md | 2 ++ docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Interpreter.Lir/LirInterpreter.fs | 5 +++++ .../ConformanceTests.fs | 3 +++ .../LirInterpreterTests.fs | 18 ++++++++++++++++++ 5 files changed, 29 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3814399..912aaf4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Executed floating-point comparisons and empty-list tail failures consistently in LIR and generated Go. + - Enforced cross-runtime equality conformance for closures, type tokens, and unequal collections. - Added short-circuit evaluation for Boolean `&&` and `||` across interpreted and generated programs. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index 7befcba..a2b38a2 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -52,6 +52,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Compiled map patterns consume entries deterministically in map order, merge repeated bindings only when structurally equal, and bind an optional tail to the remaining ordered map. Compiled type-reference patterns use resolved types rather than source text lookup. - Record updates evaluate the base once and then updates in source order. Ranges, loops, interpolation, indexing, raise, append, and cons preserve the evaluator's existing evaluation order and failures. - Equality is structural for primitives, tuples, records, maps, options, lists, unions, and type tokens, recursively. Functions, tasks, constructors, and extern handles are not structurally equal. Backend host-language comparability must not leak into this behavior. +- Integer and floating-point comparisons share the same resolved numeric operations and are executable in both compiled runtimes. Pure collection functions retain their failure behavior, including an explicit runtime error for `List.tail []`. - Cross-runtime failure conformance classifies explicit `raise`, exhausted pattern matches, and integer division by zero independently of each host runtime's exception or panic text. - Cross-runtime tests compare the legacy evaluator with artifact-round-tripped LIR execution and generated Go for successful values and failure categories. Named records passed through typed function boundaries keep record field projection semantics; their nominal annotation must not select a union runtime representation. - Cross-runtime result formatting quotes strings at every nesting level, renders unit as `()`, renders ordered maps as `map { key => value }`, uses compact semicolon separators for lists, and retains comma-space separators for tuples. diff --git a/src/FScript.Interpreter.Lir/LirInterpreter.fs b/src/FScript.Interpreter.Lir/LirInterpreter.fs index 6456bb6..724f367 100644 --- a/src/FScript.Interpreter.Lir/LirInterpreter.fs +++ b/src/FScript.Interpreter.Lir/LirInterpreter.fs @@ -161,6 +161,7 @@ module LirInterpreter = | "List.tryItem", [ VInt index; VList values ] -> if index < 0L || index >= int64 values.Length then Ok(VOption None) else Ok(VOption(Some values.[int index])) | "List.tryHead", [ VList values ] -> Ok(VOption(List.tryHead values)) | "List.tail", [ VList(_ :: tail) ] -> Ok(VList tail) + | "List.tail", [ VList [] ] -> fail "FSI3022" span "List.tail expects a non-empty list." | "List.append", [ VList left; VList right ] -> Ok(VList(left @ right)) | "List.contains", [ value; VList values ] -> Ok(VBool(List.exists (equal value) values)) | "Option.isNone", [ VOption value ] -> Ok(VBool value.IsNone) @@ -335,9 +336,13 @@ module LirInterpreter = | Lir.IntRemainder, VInt a, VInt b -> Ok(VInt(a % b)) | Lir.StructuralEqual, a, b -> Ok(VBool(equal a b)) | Lir.NumericLessThan, VInt a, VInt b -> Ok(VBool(a < b)) + | Lir.NumericLessThan, VFloat a, VFloat b -> Ok(VBool(a < b)) | Lir.NumericGreaterThan, VInt a, VInt b -> Ok(VBool(a > b)) + | Lir.NumericGreaterThan, VFloat a, VFloat b -> Ok(VBool(a > b)) | Lir.NumericLessThanOrEqual, VInt a, VInt b -> Ok(VBool(a <= b)) + | Lir.NumericLessThanOrEqual, VFloat a, VFloat b -> Ok(VBool(a <= b)) | Lir.NumericGreaterThanOrEqual, VInt a, VInt b -> Ok(VBool(a >= b)) + | Lir.NumericGreaterThanOrEqual, VFloat a, VFloat b -> Ok(VBool(a >= b)) | Lir.BooleanAnd, VBool a, VBool b -> Ok(VBool(a && b)) | Lir.BooleanOr, VBool a, VBool b -> Ok(VBool(a || b)) | _ -> fail "FSI3006" expression.Span "Primitive operands do not match their LIR operation.") diff --git a/tests/FScript.Conformance.Tests/ConformanceTests.fs b/tests/FScript.Conformance.Tests/ConformanceTests.fs index bd49d14..26fad4d 100644 --- a/tests/FScript.Conformance.Tests/ConformanceTests.fs +++ b/tests/FScript.Conformance.Tests/ConformanceTests.fs @@ -98,6 +98,7 @@ module private Harness = | Error "FSI3005" -> Some DivisionByZero | Error "FSI3012" -> Some MatchFailure | Error "FSI3011" -> Some Raised + | Error "FSI3022" -> Some Raised | Error code -> failwith $"Unexpected LIR failure category {code}" let goFailure source = @@ -155,6 +156,7 @@ type ConformanceTests() = [ value) = (fun value -> value)")>] [] [] + [ 1.5, 1.5 <= 1.5, 2.0 >= 2.0)")>] member _.``Legacy LIR artifact and Go results conform`` (source: string) = let expected = Harness.legacy source match Harness.lir source with @@ -171,6 +173,7 @@ type ConformanceTests() = [] [ 0")>] [] + [] member _.``Failure categories conform`` (source: string) = let expected = Harness.legacyFailure source Assert.That(expected.IsSome, Is.True, source) diff --git a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs index 8a15c41..5b1ac6d 100644 --- a/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs +++ b/tests/FScript.Interpreter.Lir.Tests/LirInterpreterTests.fs @@ -44,6 +44,24 @@ type LirInterpreterTests() = | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VBool expected)) | Error diagnostic -> Assert.Fail(errorText diagnostic) + [] + member _.``Executes float comparisons`` () = + match Compiler.compile { Program = Parser.parseProgram "(1.5 < 2.0, 2.0 >= 2.0)"; Externs = [] } with + | Error diagnostics -> Assert.Fail(errorText diagnostics) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok result -> Assert.That(result.Value, Is.EqualTo(LirValue.VTuple [ LirValue.VBool true; LirValue.VBool true ])) + | Error diagnostic -> Assert.Fail(errorText diagnostic) + + [] + member _.``Reports an empty List tail as a runtime error`` () = + match Compiler.compile { Program = Parser.parseProgram "List.tail []"; Externs = [] } with + | Error diagnostics -> Assert.Fail(errorText diagnostics) + | Ok compiled -> + match LirInterpreter.execute { ScriptName = None; Arguments = [] } compiled.Lir with + | Ok _ -> Assert.Fail("Expected List.tail to fail") + | Error diagnostic -> Assert.That(diagnostic.Code, Is.EqualTo("FSI3022")) + [] [] [] From 68b42f297e24f7ca598b95feeabba8dbd93e76fe Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 22:40:39 +0200 Subject: [PATCH 067/133] fix(go): preserve floating point runtime semantics --- CHANGELOG.md | 2 ++ docs/specs/compiler-semantics-baseline.md | 1 + src/FScript.Backend.Go/GoBackend.fs | 5 ++- src/FScript.Backend.Go/Runtime/runtime.go | 32 +++++++++++++++---- .../GoBackendTests.fs | 1 + .../ConformanceTests.fs | 1 + 6 files changed, 34 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 912aaf4..312e1d6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Preserved integral float representations and IEEE zero-division results in generated Go. + - Executed floating-point comparisons and empty-list tail failures consistently in LIR and generated Go. - Enforced cross-runtime equality conformance for closures, type tokens, and unequal collections. diff --git a/docs/specs/compiler-semantics-baseline.md b/docs/specs/compiler-semantics-baseline.md index a2b38a2..78e4343 100644 --- a/docs/specs/compiler-semantics-baseline.md +++ b/docs/specs/compiler-semantics-baseline.md @@ -53,6 +53,7 @@ Compiled maps use an ordered entry representation for string and integer keys. E - Record updates evaluate the base once and then updates in source order. Ranges, loops, interpolation, indexing, raise, append, and cons preserve the evaluator's existing evaluation order and failures. - Equality is structural for primitives, tuples, records, maps, options, lists, unions, and type tokens, recursively. Functions, tasks, constructors, and extern handles are not structurally equal. Backend host-language comparability must not leak into this behavior. - Integer and floating-point comparisons share the same resolved numeric operations and are executable in both compiled runtimes. Pure collection functions retain their failure behavior, including an explicit runtime error for `List.tail []`. +- Go emits integral-valued float literals with an explicit floating representation even inside erased aggregates. Floating division occurs at runtime and preserves FScript/.NET `Infinity`, `-Infinity`, and `NaN` result formatting. - Cross-runtime failure conformance classifies explicit `raise`, exhausted pattern matches, and integer division by zero independently of each host runtime's exception or panic text. - Cross-runtime tests compare the legacy evaluator with artifact-round-tripped LIR execution and generated Go for successful values and failure categories. Named records passed through typed function boundaries keep record field projection semantics; their nominal annotation must not select a union runtime representation. - Cross-runtime result formatting quotes strings at every nesting level, renders unit as `()`, renders ordered maps as `map { key => value }`, uses compact semicolon separators for lists, and retains comma-space separators for tuples. diff --git a/src/FScript.Backend.Go/GoBackend.fs b/src/FScript.Backend.Go/GoBackend.fs index d319ea7..beca409 100644 --- a/src/FScript.Backend.Go/GoBackend.fs +++ b/src/FScript.Backend.Go/GoBackend.fs @@ -76,7 +76,9 @@ module GoBackend = let private constant = function | Lir.Unit -> "struct{}{}" | Lir.Int value -> $"int64({value})" - | Lir.Float value -> value.ToString("R", Globalization.CultureInfo.InvariantCulture) + | Lir.Float value -> + let rendered = value.ToString("R", Globalization.CultureInfo.InvariantCulture) + if rendered.Contains(".") || rendered.Contains("E") || rendered.Contains("e") then rendered else rendered + ".0" | Lir.Bool true -> "true" | Lir.Bool false -> "false" | Lir.String value -> System.Text.Json.JsonSerializer.Serialize(value) @@ -261,6 +263,7 @@ module GoBackend = | Lir.Operation(op, [ left; right ]) -> match emitExpression environment left, emitExpression environment right with | Ok l, Ok r when op = Lir.StructuralEqual -> Ok $"fsruntime.Equal({l}, {r})" + | Ok l, Ok r when op = Lir.FloatDivide -> Ok $"fsruntime.FloatDivide({l}, {r})" | Ok l, Ok r -> match operation op with | Some token -> diff --git a/src/FScript.Backend.Go/Runtime/runtime.go b/src/FScript.Backend.Go/Runtime/runtime.go index 56c7e31..a23e3c8 100644 --- a/src/FScript.Backend.Go/Runtime/runtime.go +++ b/src/FScript.Backend.Go/Runtime/runtime.go @@ -2,6 +2,7 @@ package fsruntime import ( "fmt" + "math" "reflect" "sort" "strconv" @@ -65,13 +66,14 @@ func utf16Substring(value string, start int64, length int64) Option { return Some(string(utf16.Decode(units[start : start+length]))) } -func Unit(value any) struct{} { return struct{}{} } -func Int(value any) int64 { return value.(int64) } -func Float(value any) float64 { return value.(float64) } -func Bool(value any) bool { return value.(bool) } -func String(value any) string { return value.(string) } -func List(value any) []any { return value.([]any) } -func AsTuple(value any) Tuple { return value.(Tuple) } +func Unit(value any) struct{} { return struct{}{} } +func Int(value any) int64 { return value.(int64) } +func Float(value any) float64 { return value.(float64) } +func FloatDivide(left float64, right float64) float64 { return left / right } +func Bool(value any) bool { return value.(bool) } +func String(value any) string { return value.(string) } +func List(value any) []any { return value.([]any) } +func AsTuple(value any) Tuple { return value.(Tuple) } func TryRecord(value any) (map[string]any, bool) { if record, ok := value.(map[string]any); ok { return record, true @@ -395,6 +397,18 @@ func Intrinsic(name string, args []any) any { } } func Format(value any) string { return formatValue(value, false) } +func formatFloat(value float64) string { + if math.IsInf(value, 1) { + return "Infinity" + } + if math.IsInf(value, -1) { + return "-Infinity" + } + if math.IsNaN(value) { + return "NaN" + } + return strconv.FormatFloat(value, 'g', -1, 64) +} func formatValue(value any, nested bool) string { if item, ok := TryUnion(value); ok { if !item.HasValue { @@ -408,6 +422,8 @@ func formatValue(value any, nested bool) string { return "true" } return "false" + case float64: + return formatFloat(item) case string: return strconv.Quote(item) case TypeToken: @@ -473,6 +489,8 @@ func formatInterpolationValue(value any) string { return item.CaseName + " " + formatInterpolationValue(item.Value) } switch item := value.(type) { + case float64: + return formatFloat(item) case string: return item case TypeToken: diff --git a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs index 45fce66..425a03c 100644 --- a/tests/FScript.Backend.Go.Tests/GoBackendTests.fs +++ b/tests/FScript.Backend.Go.Tests/GoBackendTests.fs @@ -209,6 +209,7 @@ type GoBackendTests() = [] [] [] + [] member _.``Generated projects run with Go`` (source: string, expected: string) = requireGo () let input = { Program = Parser.parseProgram source; Externs = [] } diff --git a/tests/FScript.Conformance.Tests/ConformanceTests.fs b/tests/FScript.Conformance.Tests/ConformanceTests.fs index 26fad4d..270920d 100644 --- a/tests/FScript.Conformance.Tests/ConformanceTests.fs +++ b/tests/FScript.Conformance.Tests/ConformanceTests.fs @@ -157,6 +157,7 @@ type ConformanceTests() = [] [] [ 1.5, 1.5 <= 1.5, 2.0 >= 2.0)")>] + [] member _.``Legacy LIR artifact and Go results conform`` (source: string) = let expected = Harness.legacy source match Harness.lir source with From 20677d320e01427782418e55129264a3ff525635 Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 22:51:45 +0200 Subject: [PATCH 068/133] docs(compiler): specify HIR LIR and execution workflows --- CHANGELOG.md | 2 + docs/README.md | 2 + docs/guides/compile-to-go.md | 134 ++++++++++++++++++++++++++++++ docs/guides/compiler-cli.md | 2 + docs/guides/interpret-ir.md | 132 +++++++++++++++++++++++++++++ docs/specs/README.md | 2 + docs/specs/executable-lir.md | 155 +++++++++++++++++++++++++++++++++++ docs/specs/typed-hir.md | 139 +++++++++++++++++++++++++++++++ 8 files changed, 568 insertions(+) create mode 100644 docs/guides/compile-to-go.md create mode 100644 docs/guides/interpret-ir.md create mode 100644 docs/specs/executable-lir.md create mode 100644 docs/specs/typed-hir.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 312e1d6..abe9aae 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Documented the typed HIR, executable LIR, Go compilation workflow, and IR interpreter API. + - Preserved integral float representations and IEEE zero-division results in generated Go. - Executed floating-point comparisons and empty-list tail failures consistently in LIR and generated Go. diff --git a/docs/README.md b/docs/README.md index 2f0382c..e568456 100644 --- a/docs/README.md +++ b/docs/README.md @@ -11,6 +11,8 @@ This folder is organized into three layers: - Tutorial: [`guides/getting-started-tutorial.md`](./guides/getting-started-tutorial.md) - REPL guide: [`guides/repl.md`](./guides/repl.md) - Compiler CLI: [`guides/compiler-cli.md`](./guides/compiler-cli.md) +- Compile to Go: [`guides/compile-to-go.md`](./guides/compile-to-go.md) +- Compile and interpret IR: [`guides/interpret-ir.md`](./guides/interpret-ir.md) - FScript vs F# / OCaml: [`guides/fsharp-ocaml-differences.md`](./guides/fsharp-ocaml-differences.md) - Language rationale: [`guides/language-choices.md`](./guides/language-choices.md) diff --git a/docs/guides/compile-to-go.md b/docs/guides/compile-to-go.md new file mode 100644 index 0000000..bc5c344 --- /dev/null +++ b/docs/guides/compile-to-go.md @@ -0,0 +1,134 @@ +# Compile FScript to Go + +The Go target turns one complete FScript program into an inspectable Go project. Source generation does not invoke the Go toolchain. + +## Prerequisites + +- the `fscript` CLI built from a compiler-enabled FScript distribution; +- Go 1.25 or newer to format, test, or run the generated project. + +Install the CLI from the repository when working on this unreleased compiler surface: + +```bash +dotnet tool restore +dotnet build FScript.sln -c Release +``` + +## Generate a project + +Given `hello.fss`: + +```fsharp +let greet name = $"Hello, {name}!" + +match Env.Arguments with +| name :: _ -> greet name +| [] -> greet "world" +``` + +generate Go source with: + +```bash +fscript compile hello.fss --target go --output ./out/hello-go +``` + +When running from the repository instead of an installed tool: + +```bash +dotnet run --project src/FScript -- \ + compile hello.fss --target go --output ./out/hello-go +``` + +The output is a complete project: + +```text +out/hello-go/ + go.mod + main.go + internal/ + fscriptprogram/ + program.go + fsruntime/ + closures.go + environment.go + patterns.go + runtime.go +``` + +The project declares `go 1.25`, contains no Go generic declarations, and includes the FScript-owned runtime helpers it needs. + +## Inspect and run + +```bash +cd out/hello-go +gofmt -d . +go test ./... +go run . Ada +``` + +Expected output: + +```text +"Hello, Ada!" +``` + +Generated source is already `gofmt`-compatible, so `gofmt -d .` should print nothing. `Env.Arguments` receives executable arguments. In generated Go, `Env.ScriptName` is the generated executable name rather than the original `.fss` path. + +## Semantics and representations + +The backend consumes validated LIR; it does not type-check source or implement pattern semantics independently. + +The initial backend preserves: + +- closures, currying, partial application, and recursive groups; +- tuples and concrete structs for declared records; +- concrete tagged structures for declared unions; +- structural/named record conversions; +- erased polymorphic lists, options, maps, and closures; +- ordered maps and recursive structural equality; +- pattern decision plans and guards; +- scalar, string, list, option, map, and portable console intrinsics; +- short-circuit Boolean operations and left-to-right evaluation; +- FScript-compatible result formatting, including IEEE floating results. + +The backend deliberately does not use Go generics. + +## Go imports + +A directly representable Go function can be declared without a body: + +```fsharp +[] +let upper : string -> string + +upper "hello" +``` + +The initial function boundary accepts unit and primitive arguments/results, plus supported imported record types. Imported records require exported, exactly named primitive fields: + +```fsharp +[] +type Customer = { + Name: string + Seats: int +} +``` + +FScript does not inspect Go packages or add module versions. For non-standard-library imports, the generated project owner must configure dependencies with ordinary Go tooling, for example `go get` or an edited `go.mod`. + +Unsupported generic types, channels, interfaces, embedded fields, function fields, complex pointers/maps, imported unions, and non-trivial adapters produce `FSB4xxx` diagnostics. + +## Unsupported runtime features + +Reachable Tasks and arbitrary host externs are not part of the initial Go runtime. Generation reports an explicit backend diagnostic. An unused unsupported declaration does not prevent generation. + +The CLI prints compiler diagnostics and exits non-zero. Common diagnostic families are: + +- `FSH1xxx`: typed HIR/frontend compilation; +- `FSL2xxx`: LIR lowering or validation; +- `FSB4xxx`: Go representation or linkage; +- `FSA5xxx`: output/artifact filesystem failures. + +## Repeated generation + +Running the same command again deterministically replaces the files owned by the generator. The backend does not run `go build`, manage a cache, or produce a binary. diff --git a/docs/guides/compiler-cli.md b/docs/guides/compiler-cli.md index ccb3739..658ada5 100644 --- a/docs/guides/compiler-cli.md +++ b/docs/guides/compiler-cli.md @@ -12,3 +12,5 @@ The IR form writes `/.fsir`. The Go form writes a proje Both forms create missing parent directories and deterministically replace files owned by the selected output on a repeated invocation. An unknown target is rejected before the output directory is created. Filesystem failures are reported as `FSA5101` diagnostics. Generated projects declare `go 1.25`, use no Go generics, and are already formatted for an empty `gofmt -d` result. Generation itself does not invoke Go; building or running the resulting project remains the caller's responsibility. + +See the complete workflows in [Compile FScript to Go](./compile-to-go.md) and [Compile and Interpret FScript IR](./interpret-ir.md). The normative representations are documented in the [typed HIR](../specs/typed-hir.md) and [executable LIR](../specs/executable-lir.md) specifications. diff --git a/docs/guides/interpret-ir.md b/docs/guides/interpret-ir.md new file mode 100644 index 0000000..f8ffab1 --- /dev/null +++ b/docs/guides/interpret-ir.md @@ -0,0 +1,132 @@ +# Compile and Interpret FScript IR + +FScript can compile one complete source program into a versioned `.fsir` artifact and execute its LIR from .NET without returning to parser AST evaluation. + +## Produce an artifact + +```bash +fscript compile program.fss --target ir --output ./out +``` + +For `program.fss`, this writes: + +```text +out/program.fsir +``` + +From the repository: + +```bash +dotnet run --project src/FScript -- \ + compile program.fss --target ir --output ./out +``` + +The artifact is canonical indented JSON. It contains the complete validated LIR program, format/version metadata, compiler and language versions, source hashes for file-backed inputs, and ordered foreign dependencies. + +Treat `.fsir` as an internal build artifact. Version 1 has no backward-compatibility guarantee; use matching compiler and interpreter builds. + +## Reference the interpreter + +The compiler and interpreter are separate assemblies. In a repository-local .NET project, add: + +```xml + + + + +``` + +`FScript.Compiler` owns LIR and artifact serialization. `FScript.Interpreter.Lir` owns execution. Both APIs currently use the `FScript.Compiler` namespace. + +## Load and execute + +```fsharp +open System +open System.IO +open FScript.Compiler + +let artifact = File.ReadAllBytes "out/program.fsir" + +match IrArtifact.deserialize artifact with +| Error diagnostics -> + for diagnostic in diagnostics do + eprintfn "%s: %s" diagnostic.Code diagnostic.Message + 1 +| Ok program -> + let context = + { ScriptName = Some "program.fss" + Arguments = [ "first"; "second" ] } + + match LirInterpreter.execute context program with + | Error diagnostic -> + eprintfn "%s: %s" diagnostic.Code diagnostic.Message + 1 + | Ok execution -> + for line in execution.Stdout do + printfn "%s" line + + printfn "Result: %A" execution.Value + 0 +``` + +`ExecutionResult.Stdout` contains `Console.writeLine` output in execution order. `ExecutionResult.Value` is an `LirValue`, with cases for primitives, tuples, lists, records, options, unions, ordered maps, type tokens, and callable/runtime handles. + +Applications should pattern-match `LirValue` when they need a stable domain conversion instead of relying on `%A` formatting. + +## Compile in memory + +Source can also be compiled and executed without writing `.fsir`: + +```fsharp +open FScript.Compiler +open FScript.Language + +let input = + { Program = Parser.parseProgram "let add x y = x + y\nadd 1 2" + Externs = [] } + +match Compiler.compile input with +| Error diagnostics -> Error diagnostics +| Ok compiled -> + LirInterpreter.execute + { ScriptName = None; Arguments = [] } + compiled.Lir + |> Result.mapError List.singleton +``` + +Use `parseFileWithIncludes` through the normal host/CLI path when compiling file-backed programs with include resolution. `Parser.parseProgram` parses only the supplied source text. + +## Validation and failures + +Deserialization performs all of these checks before returning a program: + +1. JSON and required metadata are valid. +2. `format` is `fscript-lir`. +3. `irVersion` is supported. +4. DTO fields reconstruct a complete LIR program. +5. `LirValidation.validate` accepts the reconstructed program. + +`LirInterpreter.execute` validates caller-created LIR again before execution. + +Diagnostic families are: + +- `FSA5xxx`: artifact format, version, metadata, or reconstruction; +- `FSL2xxx`: malformed or inconsistent LIR; +- `FSI3xxx`: execution failure or unsupported reachable linkage. + +An executed `raise`, failed match, integer division by zero, or invalid collection operation returns a structured interpreter diagnostic. The interpreter rejects reachable target imports, Tasks, and arbitrary host externs. Unused unsupported declarations remain inert. + +## Runtime context + +`InterpreterContext` supplies the compiler runtime-context intrinsic: + +```fsharp +type Environment = { + ScriptName: string option + Arguments: string list +} + +let Env : Environment +``` + +The interpreter does not read process arguments implicitly. The embedding host decides exactly which script name and arguments the program receives. diff --git a/docs/specs/README.md b/docs/specs/README.md index 284cf8b..a0752b7 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -5,6 +5,8 @@ Normative behavior for the language, runtime surface, hosting model, and editor/ ## Language and type system - Compiler semantic baseline: [`compiler-semantics-baseline.md`](./compiler-semantics-baseline.md) +- Typed high-level IR (HIR): [`typed-hir.md`](./typed-hir.md) +- Executable low-level IR (LIR): [`executable-lir.md`](./executable-lir.md) - Syntax and indentation: [`syntax-and-indentation.md`](./syntax-and-indentation.md) - Supported types: [`supported-types.md`](./supported-types.md) - Function annotations: [`function-annotations.md`](./function-annotations.md) diff --git a/docs/specs/executable-lir.md b/docs/specs/executable-lir.md new file mode 100644 index 0000000..ce3618f --- /dev/null +++ b/docs/specs/executable-lir.md @@ -0,0 +1,155 @@ +# Executable Low-Level IR (LIR) + +## Status and purpose + +This document specifies FScript's executable low-level intermediate representation. LIR is backend-independent, structured, non-SSA, and suitable for both direct interpretation and source generation. + +LIR is produced only from validated HIR: + +```text +Hir.Program + -> LirLowering.lower + -> LirOptimization.optimize + -> LirValidation.validate + -> Lir.Program +``` + +The version-1 optimization pass is intentionally identity-only. LIR and the JSON DTO schema used by `.fsir` are internal versioned contracts, not stable public interchange standards. + +## Program model + +`Lir.Program` contains: + +- concrete named type declarations; +- imported record linkage and field schemas; +- lifted functions; +- ordered top-level declarations; +- an optional entry expression. + +`SymbolId` identifies values and bindings. `FunctionId` independently identifies lifted function bodies. Both are deterministic artifact-local integers. + +Every expression carries its semantic FScript type and source span. Rich types remain available for validation and backend representation selection even where runtime values are erased. + +## Functions and closures + +Every source lambda is lifted into a `Lir.Function` containing: + +- one parameter ID and type; +- an ordered, explicit capture list; +- a return type; +- a source span; +- a body expression. + +`MakeClosure(functionId, captures)` constructs a closure. `CallClosure` applies one argument. `PipeClosure` preserves value-first pipeline evaluation. Partial application follows naturally from nested unary closures. + +`LetRecursiveGroup` establishes all recursive values before their bodies run. Lifted functions validate their capture list against the ordered free-symbol set of their body. + +## Operations and control flow + +LIR uses explicit operations for integer and floating arithmetic, integer remainder, structural equality, numeric comparisons, and Boolean operations. + +Control flow remains structured: + +- `Branch` represents `if`; +- `Let` and `LetRecursiveGroup` represent lexical scope; +- `Fail` represents `raise`; +- `ForEach` represents ordered list iteration; +- `DecisionTree` represents pattern matching. + +Operands normally evaluate left to right and stop at the first failure. `BooleanAnd` and `BooleanOr` short-circuit. Branches evaluate only the selected body. + +## Runtime values and aggregates + +LIR directly constructs and projects: + +- unit, integer, float, Boolean, and string constants; +- tuples and lists; +- options; +- records and record updates; +- declared tagged unions; +- ordered maps; +- type tokens. + +Named records and unions retain declarations in the program type table. `RecordConvert` carries frontend-proven structural conversion evidence between distinct representations. + +Polymorphic lists, options, maps, and closures may use erased runtime payloads. LIR does not require or assume Go generics. + +Maps are ordered string-key maps according to current FScript semantics. Consumers must not substitute host-language hash-map iteration order. + +Structural equality recursively covers primitives, tuples, records, maps, options, lists, unions, and type tokens. Closures, task values, constructors, and extern handles are not structurally equal. + +## Pattern decision plans + +Pattern lowering produces a `DecisionTree` whose target expression is evaluated exactly once. Each case contains a `PatternPlan`, optional guard, body, and source span. + +A plan consists of: + +- projection steps such as list head/tail, tuple item, record field, option value, and union payload; +- checks for literals, list shape, tuple arity, record fields, options, union cases, and type shape; +- symbol bindings associated with deterministic projections; +- ordered map matching plans, including remainder handling. + +Consumers execute this plan; the Go backend does not reimplement source-pattern semantics. + +Cases are attempted in source order. Bindings become visible only after a plan succeeds. Guards run after bindings and before the body. Exhausting every case produces the standard match failure. + +## Linkage and reachability + +LIR retains the same explicit linkage categories as HIR: + +- portable `Intrinsic` values; +- `RuntimeContext` for `Env`; +- `HostExternal` handles; +- `TargetImport` handles. + +The LIR interpreter rejects a reachable fully-applied target import, Task operation, or unsupported host external with an `FSI3xxx` diagnostic. The Go backend rejects reachable Task or unsupported host-external linkage with `FSB4xxx`. Unused unsupported declarations remain inert. + +## Validation + +`LirValidation.validate` returns `Result` and checks at least: + +- unique symbol and function IDs; +- known named types and valid recursive type references; +- valid nested spans; +- resolved loads, captures, and function IDs; +- exact closure capture lists and call signatures; +- primitive arity and operand/result types; +- aggregate, projection, update, and conversion representations; +- decision-plan checks, bindings, and map projections; +- declaration and function result types; +- known intrinsic names and complete linkage metadata. + +LIR/lowering diagnostics use `FSL2xxx` codes. + +Both `LirInterpreter.execute` and `GoBackend.generate` validate caller-provided LIR before consuming it. + +## Artifact format + +`IrArtifact.serialize` writes one complete `Lir.Program` as canonical indented UTF-8 JSON. The artifact includes: + +- `format = "fscript-lir"`; +- `irVersion = 1`; +- compiler and language versions; +- normalized source paths and SHA-256 hashes when file-backed sources are available; +- ordered foreign dependencies; +- the complete LIR program. + +`IrArtifact.deserialize` rejects an unknown format or version before constructing LIR, validates required metadata, reconstructs the program through explicit DTO parsing, and runs LIR validation before returning it. Artifact diagnostics use `FSA5xxx` codes. + +No backward-compatibility guarantee is made for version 1. Producers and consumers should use matching FScript builds. + +## Execution APIs + +The principal APIs are: + +```fsharp +IrArtifact.serialize : Lir.Program -> byte array +IrArtifact.deserialize : byte array -> Result + +LirInterpreter.execute : + InterpreterContext -> Lir.Program -> Result +``` + +`InterpreterContext` supplies an optional script name and argument list. `ExecutionResult` contains the final `LirValue` and ordered `Console.writeLine` output. + +`Lir.format : Lir.Program -> string` is a deterministic debug/snapshot representation and is not the artifact encoding. diff --git a/docs/specs/typed-hir.md b/docs/specs/typed-hir.md new file mode 100644 index 0000000..f4ab8a2 --- /dev/null +++ b/docs/specs/typed-hir.md @@ -0,0 +1,139 @@ +# Typed High-Level IR (HIR) + +## Status and purpose + +This document specifies the high-level intermediate representation produced by `FScript.Compiler`. HIR is the first compiler-owned form after parsing, name resolution, and Hindley–Milner inference. It is fully resolved, fully typed, platform-neutral, and still preserves FScript language structure. + +HIR is an internal compiler contract. It is not the `.fsir` artifact format and has no compatibility guarantee between FScript versions. + +The public construction path is: + +```text +Program + -> TypeInfer.inferResolvedProgramWithExterns + -> HirLowering.lower + -> HirValidation.validate + -> Hir.Program +``` + +Callers normally use `Compiler.compile`, which performs this pipeline and also produces validated LIR. + +## Program model + +`Hir.Program` contains, in source-semantic order: + +- `Types`: named record and union declarations; +- `ImportedRecords`: Go-imported record representation declarations; +- `Declarations`: top-level values and functions; +- `Entry`: the optional final top-level expression. + +Includes are resolved before HIR construction. One `Hir.Program` therefore represents the complete source program rather than one source file. + +Every type, declaration, parameter, pattern binding, local binding, builtin, intrinsic, and extern reference uses a deterministic program-local `SymbolId`. IDs identify declarations; names are retained for diagnostics and readable debug output but never define binding identity. + +## Type information + +Every `Hir.Expression` and `Hir.Pattern` carries: + +- its final instantiated FScript `Type`; +- its originating `Span`; +- a semantic node. + +Top-level declarations additionally retain their normalized `Scheme`. Type-variable numbering is deterministic and independent of process-global inference counters. + +HIR preserves: + +- primitive, tuple, list, option, map, function, task, record, union, named, and type-token types; +- named type identity; +- structural record shapes; +- recursive type references; +- polymorphic declaration schemes; +- explicit named/structural record conversion evidence through `RecordConvert`. + +A backend must not repeat FScript compatibility checking or infer structural conversions from its host representation. + +## Declarations and metadata + +A value declaration contains its symbol ID, source name, normalized scheme, recursive/export flags, structured attributes, span, and value expression. + +`Export` metadata records the exported source name and attribute span. Other attributes retain their name, ordered string arguments, and span. + +A type declaration contains its symbol ID, name, recursion flag, complete semantic definition, attributes, optional import metadata, and span. Imported records remain ordinary FScript record types; import metadata changes their backend representation, not their FScript typing rules. + +## Expressions + +HIR directly represents: + +- constants and resolved references; +- lambdas, application, and value-first pipelines; +- `if`, local `let`, recursive bindings, and mutually recursive groups; +- resolved primitive operations; +- tuples, records, record updates, explicit record conversions, and field access; +- lists, options, maps, map spread, ranges, cons, append, and indexing; +- declared union construction; +- interpolation, `raise`, `for`, `typeof`, and matches; +- portable intrinsics, unsupported host extern handles, runtime context, and target imports. + +Primitive nodes distinguish integer and floating arithmetic. Structural equality and numeric/Boolean operations are resolved before HIR so consumers never dispatch an operator by source spelling. + +Function application is unary and curried. A multi-argument source function is represented as nested lambdas and applications. + +## Patterns and matches + +HIR retains FScript patterns before decision-tree lowering: + +- wildcard and binding patterns; +- literal patterns; +- empty/cons list patterns; +- tuple and record patterns; +- ordered map clauses and optional remainder patterns; +- option and declared-union patterns; +- runtime type-reference patterns. + +A match case contains a pattern, optional Boolean guard, body, and case span. Cases and guards retain source order. Pattern bindings use the same deterministic IDs as references in guards and bodies. + +## Evaluation semantics + +HIR child order is semantic: + +- declarations and top-level expressions execute in source order; +- expression operands and aggregate elements evaluate left to right; +- failures stop subsequent evaluation; +- `&&` and `||` short-circuit their right operand; +- a match scrutinee is evaluated once; +- cases and guards are attempted in source order; +- a non-recursive binding initializer does not see its new binding; +- a recursive group sees every member of that group. + +## Linkage + +Compiler linkage is declarative and contains no executable .NET delegate: + +- `Intrinsic name` identifies a supported portable builtin; +- `RuntimeContext` identifies `Env`; +- `HostExternal(name, arity)` identifies a host-only extern; +- `TargetImport(modulePath, memberName)` identifies backend-provided code. + +Unsupported linkage remains representable in HIR so reachability-sensitive consumers can reject only executed or generated uses. + +## Validation + +`HirValidation.validate` returns `Result` and enforces at least: + +- unique type and value symbol IDs; +- valid spans, including nested expressions and patterns; +- known named types and union cases; +- resolved references and pattern bindings; +- expression, branch, call, aggregate, and declaration type consistency; +- valid recursive groups and declaration schemes; +- valid primitive signatures; +- valid record conversion evidence; +- complete import and export metadata. + +HIR diagnostics use `FSH1xxx` codes and preserve an originating FScript span whenever one exists. + +## Deterministic debugging + +`Hir.format : Hir.Program -> string` produces a canonical, human-readable representation with stable ordering, symbol IDs, and normalized types. It is intended for diagnostics and snapshot tests, not serialization. + +The compiler serializes validated LIR, not formatted HIR. From c38d63730d5e74322ff7be772ddc51c0e82431de Mon Sep 17 00:00:00 2001 From: Pierre Chalamet Date: Sun, 23 Aug 2026 23:01:44 +0200 Subject: [PATCH 069/133] docs(website): add Go and IR compiler paths --- CHANGELOG.md | 2 + website/docs/compiler/compile-to-go.md | 75 +++++++++++++++++++++++++ website/docs/compiler/hir.md | 54 ++++++++++++++++++ website/docs/compiler/interpret-ir.md | 62 ++++++++++++++++++++ website/docs/compiler/lir.md | 47 ++++++++++++++++ website/docs/compiler/overview.md | 48 ++++++++++++++++ website/docs/reference/cli-reference.md | 2 + website/docs/whats-new.md | 2 +- website/docusaurus.config.ts | 7 ++- website/sidebars.ts | 11 ++++ website/src/pages/index.module.css | 24 +++++++- website/src/pages/index.tsx | 48 +++++++++++++--- 12 files changed, 371 insertions(+), 11 deletions(-) create mode 100644 website/docs/compiler/compile-to-go.md create mode 100644 website/docs/compiler/hir.md create mode 100644 website/docs/compiler/interpret-ir.md create mode 100644 website/docs/compiler/lir.md create mode 100644 website/docs/compiler/overview.md diff --git a/CHANGELOG.md b/CHANGELOG.md index abe9aae..d101184 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,8 @@ All notable changes to FScript are documented in this file. ## [Unreleased] +- Added website documentation and homepage paths for Go compilation and LIR interpretation. + - Documented the typed HIR, executable LIR, Go compilation workflow, and IR interpreter API. - Preserved integral float representations and IEEE zero-division results in generated Go. diff --git a/website/docs/compiler/compile-to-go.md b/website/docs/compiler/compile-to-go.md new file mode 100644 index 0000000..38bc80c --- /dev/null +++ b/website/docs/compiler/compile-to-go.md @@ -0,0 +1,75 @@ +--- +id: compile-to-go +title: Compile to Go +slug: /compiler/go +--- + +Generate a complete, inspectable Go project from one FScript program: + +```bash +fscript compile hello.fss --target go --output ./out/hello-go +``` + +## Example + +```fsharp title="hello.fss" +let greet name = $"Hello, {name}!" + +match Env.Arguments with +| name :: _ -> greet name +| [] -> greet "world" +``` + +The generated project contains: + +```text +go.mod +main.go +internal/fscriptprogram/program.go +internal/fsruntime/closures.go +internal/fsruntime/environment.go +internal/fsruntime/patterns.go +internal/fsruntime/runtime.go +``` + +It declares Go 1.25, uses no Go generics, and is already `gofmt`-compatible. + +## Build and run + +```bash +cd out/hello-go +gofmt -d . +go test ./... +go run . Ada +``` + +Output: + +```text +"Hello, Ada!" +``` + +Source generation itself does not require or invoke Go. The caller decides whether to inspect, test, run, or build the project. + +## Supported semantics + +The backend preserves closures, currying, recursion, concrete declared records and unions, structural conversions, patterns and guards, ordered maps, structural equality, erased polymorphic containers, portable builtins, `Console.writeLine`, and `Env`. + +Generated code preserves FScript evaluation order and formatting rather than inheriting accidental Go behavior. + +## Import Go functions + +Directly representable functions can be declared without a body: + +```fsharp +[] +let upper : string -> string + +upper "hello" +``` + +Imported records must have exported, exactly named primitive fields. FScript does not inspect packages or invent module versions; configure non-standard dependencies with ordinary Go tooling. + +Reachable Tasks, arbitrary host externs, unsupported imported shapes, and complex adapters produce `FSB4xxx` diagnostics. Unused unsupported declarations remain inert. + +See the [CLI reference](../reference/cli-reference.md) and the repository's [complete Go guide](https://github.com/MagnusOpera/FScript/blob/main/docs/guides/compile-to-go.md). diff --git a/website/docs/compiler/hir.md b/website/docs/compiler/hir.md new file mode 100644 index 0000000..d4eb24f --- /dev/null +++ b/website/docs/compiler/hir.md @@ -0,0 +1,54 @@ +--- +id: compiler-hir +title: Typed HIR Specification +slug: /compiler/hir +--- + +The high-level IR is the first compiler-owned form after parsing, name resolution, and Hindley–Milner inference. + +```text +Program + -> inferResolvedProgramWithExterns + -> HirLowering.lower + -> HirValidation.validate + -> Hir.Program +``` + +## Contract + +HIR is fully typed and resolved, independent of parser AST nodes and backend representations, identified by deterministic program-local symbol IDs, and validated before LIR lowering. + +Every expression and pattern carries its final FScript type and source span. Top-level declarations retain normalized polymorphic schemes, attributes, recursion metadata, and structured export metadata. + +## Program contents + +`Hir.Program` contains ordered type declarations, imported-record declarations, value declarations, and an optional final entry expression. Includes have already been resolved, so the program represents the complete compilation unit. + +HIR retains lambdas and applications, branches and lexical bindings, records and unions, lists/options/maps, interpolation, ranges, loops, matches, and type tokens. + +Operators are semantic operations such as integer addition, floating division, structural equality, or Boolean conjunction. Explicit `RecordConvert` nodes carry frontend-proven structural conversion evidence. + +## Patterns + +HIR retains wildcard, binding, literal, list, tuple, record, map, option, union, and type patterns. Match cases preserve guard and case order. Pattern bindings use the same IDs as references in guards and bodies. + +Pattern decision plans do not appear until HIR is lowered to LIR. + +## Evaluation requirements + +- Top-level declarations execute in source order. +- Children and arguments evaluate left to right. +- `&&` and `||` short-circuit. +- Match scrutinees evaluate once. +- Cases and guards run in source order. +- Recursive groups see all peers; ordinary initializers see only preceding scope. + +## Linkage and validation + +HIR distinguishes portable intrinsics, the `Env` runtime context, target imports, and host-only extern handles. Linkage descriptions carry types and arity but no executable .NET delegates. + +Validation checks unique IDs, known types, resolved references, valid spans, expression and scheme consistency, primitive signatures, recursive groups, conversion evidence, and import/export metadata. Failures use `FSH1xxx` diagnostics. + +`Hir.format` produces a deterministic debug form for snapshots. It is not an artifact format; `.fsir` stores validated LIR. + +Read the repository's [complete normative HIR specification](https://github.com/MagnusOpera/FScript/blob/main/docs/specs/typed-hir.md). diff --git a/website/docs/compiler/interpret-ir.md b/website/docs/compiler/interpret-ir.md new file mode 100644 index 0000000..6cad416 --- /dev/null +++ b/website/docs/compiler/interpret-ir.md @@ -0,0 +1,62 @@ +--- +id: interpret-ir +title: Interpret IR +slug: /compiler/interpret-ir +--- + +Compile a complete FScript program to versioned LIR: + +```bash +fscript compile program.fss --target ir --output ./out +``` + +This writes `out/program.fsir`, a canonical JSON artifact containing metadata and validated executable LIR. + +## Load from .NET + +Reference both repository-local assemblies: + +```xml + + +``` + +Then deserialize and execute: + +```fsharp +open System.IO +open FScript.Compiler + +match File.ReadAllBytes "out/program.fsir" |> IrArtifact.deserialize with +| Error diagnostics -> + diagnostics + |> List.iter (fun diagnostic -> + eprintfn "%s: %s" diagnostic.Code diagnostic.Message) +| Ok program -> + let context = + { ScriptName = Some "program.fss" + Arguments = [ "first"; "second" ] } + + match LirInterpreter.execute context program with + | Error diagnostic -> + eprintfn "%s: %s" diagnostic.Code diagnostic.Message + | Ok execution -> + execution.Stdout |> List.iter (printfn "%s") + printfn "Result: %A" execution.Value +``` + +`ExecutionResult` contains the final typed `LirValue` and ordered `Console.writeLine` output. The host supplies `Env.ScriptName` and `Env.Arguments` through `InterpreterContext`; execution does not read process arguments implicitly. + +## Safety checks + +Deserialization checks the artifact format and version, required metadata, DTO structure, and all LIR invariants. Execution validates caller-created LIR again. + +- `FSA5xxx`: artifact failures; +- `FSL2xxx`: invalid LIR; +- `FSI3xxx`: execution failures. + +Reachable target imports, Tasks, and arbitrary host externs are rejected explicitly. Pure core-language programs, portable builtins, and `Console.writeLine` execute directly from LIR. + +Version-1 `.fsir` is internal and does not promise backward compatibility. Use matching compiler and interpreter builds. + +See the [LIR specification](./lir.md) and the repository's [complete interpreter guide](https://github.com/MagnusOpera/FScript/blob/main/docs/guides/interpret-ir.md). diff --git a/website/docs/compiler/lir.md b/website/docs/compiler/lir.md new file mode 100644 index 0000000..ce5091e --- /dev/null +++ b/website/docs/compiler/lir.md @@ -0,0 +1,47 @@ +--- +id: compiler-lir +title: Executable LIR Specification +slug: /compiler/lir +--- + +LIR is FScript's backend-independent executable representation. It is structured, non-SSA, and does not assume Go syntax or Go generics. + +```text +validated HIR + -> LirLowering.lower + -> LirOptimization.optimize + -> LirValidation.validate + -> Lir.Program +``` + +The version-1 optimization pass is deliberately identity-only. + +## Program and closures + +`Lir.Program` contains named type declarations, imported-record linkage, lifted functions, ordered top-level declarations, and an optional entry expression. Every expression retains a semantic type and source span. + +Lambdas become unary lifted functions with explicit ordered captures. `MakeClosure` constructs callable values and `CallClosure` applies one argument, preserving currying and partial application. Recursive groups establish every peer closure before use. + +## Data and operations + +LIR directly represents primitive operations, branches, bindings, failures, tuples, lists, options, records, record conversions, tagged unions, ordered maps, ranges, loops, indexing, interpolation, type tokens, and linkage handles. + +Polymorphic containers and closures may use erased runtime payloads, while declared records and unions retain enough information for concrete Go representations. + +Structural equality recursively covers data values and type tokens. Functions, Tasks, constructors, and extern handles are not structurally equal. + +## Pattern plans + +Source patterns lower to deterministic decision plans. The target evaluates once; projections extract values; checks test shape and cases; bindings receive stable IDs; map clauses retain ordered matching and remainder behavior. + +The interpreter and Go runtime execute the same plan. The Go emitter does not recreate source-pattern logic. + +## Validation and artifacts + +LIR validation checks IDs, named types, spans, loads, captures, calls, operations, aggregate representations, decision plans, imports, intrinsics, and declaration/function result types. Failures use `FSL2xxx` diagnostics. + +The `.fsir` serializer writes canonical JSON containing `format = "fscript-lir"`, `irVersion = 1`, compiler/language versions, source hashes, ordered dependencies, and the complete program. Artifact failures use `FSA5xxx`. + +Both execution and Go generation validate caller-provided LIR. Version 1 is internal and has no backward-compatibility guarantee. + +Read the repository's [complete normative LIR specification](https://github.com/MagnusOpera/FScript/blob/main/docs/specs/executable-lir.md). diff --git a/website/docs/compiler/overview.md b/website/docs/compiler/overview.md new file mode 100644 index 0000000..634b3ac --- /dev/null +++ b/website/docs/compiler/overview.md @@ -0,0 +1,48 @@ +--- +id: compiler-overview +title: Compiler Overview +slug: /compiler/overview +--- + +FScript has a typed compiler pipeline in addition to its established embedded evaluators: + +```text +source + -> resolved typed frontend + -> validated HIR + -> executable LIR + -> .fsir artifact + |-> LIR interpreter + `-> Go source backend +``` + +The frontend remains FScript-specific. Go consumes backend-independent LIR; it does not introduce a second set of language typing rules. + +## What you can do + +- Compile a complete `.fss` program into one versioned `.fsir` artifact. +- Deserialize and execute LIR from the separate .NET interpreter assembly. +- Generate a complete Go 1.25 project without Go generics. +- Use directly representable Go imports through explicit `[ p { margin: 0 0 .4rem; color: #929b94; line-height: 1.7; } +.compilerSection .sectionKicker { color: #778079; } +.compilerPipeline { display: flex; gap: .8rem; align-items: center; justify-content: center; margin-top: 4rem; padding: 1.25rem; border: 1px solid #303832; background: #121814; font: 750 .69rem/1.3 var(--ifm-font-family-monospace); letter-spacing: .08em; text-transform: uppercase; } +.compilerPipeline span, .compilerPipeline b { padding: .75rem .9rem; border: 1px solid #39423b; color: #bcc5bd; background: #171e19; } +.compilerPipeline b { border-color: rgba(98,230,167,.42); color: var(--home-mint); background: rgba(98,230,167,.07); } +.compilerPipeline i { color: #667168; font-style: normal; } +.compilerPipeline .pipelineOr { color: #8e978f; text-transform: lowercase; } +.compilerCards { display: grid; grid-template-columns: repeat(2, 1fr); gap: 1.1rem; margin-top: 1.1rem; } +.compilerCard { display: flex; min-height: 310px; flex-direction: column; padding: 2rem; border: 1px solid #303832; color: #f6f8f4 !important; background: #141a16; text-decoration: none !important; transition: transform .2s ease, border-color .2s ease, box-shadow .2s ease; } +.compilerCard:hover { transform: translateY(-5px); border-color: var(--home-mint); box-shadow: 8px 9px 0 rgba(98,230,167,.1); } +.compilerCard small { color: var(--home-mint); font: 800 .68rem/1 var(--ifm-font-family-monospace); letter-spacing: .14em; } +.compilerCard h3 { margin: 2rem 0 .8rem; color: #fff; font-size: 2rem; letter-spacing: -.045em; } +.compilerCard p { max-width: 40rem; color: #929b94; line-height: 1.65; } +.compilerCard b { display: flex; justify-content: space-between; margin-top: auto; padding-top: 1rem; border-top: 1px solid #343c36; color: var(--home-mint); font-size: .78rem; text-transform: uppercase; } + .learningSection { padding: clamp(6rem, 10vw, 10rem) 0; } .learningHeader { display: grid; grid-template-columns: 1.2fr .55fr; gap: 4rem; align-items: end; } .learningHeader > p { margin: 0 0 .4rem; color: #636760; line-height: 1.7; } @@ -404,9 +423,10 @@ .heroCopy { max-width: 48rem; } .heroVisual { margin-top: 4rem; } .mascotMark { right: 1rem; } - .statementGrid, .sectionHeading, .boundaryGrid, .learningHeader { grid-template-columns: 1fr; } + .statementGrid, .sectionHeading, .boundaryGrid, .compilerHeader, .learningHeader { grid-template-columns: 1fr; } .statementGrid, .sectionHeading { gap: 1rem; } .metricStrip, .exampleGrid, .learningGrid { grid-template-columns: 1fr; } + .compilerPipeline { flex-wrap: wrap; } .metricStrip > div { border-right: 0; border-bottom: 1px solid #c8c6bd; } .metricStrip > div:last-child { border-bottom: 0; } .exampleCard > p { min-height: 0; } @@ -431,6 +451,8 @@ .codeArea pre { font-size: .66rem; } .heroTicker div { justify-content: flex-start; } .statementTitle, .sectionHeading h2, .boundaryCopy h2, .learningHeader h2, .ctaInner h2 { font-size: 2.65rem; } + .compilerHeader h2 { font-size: 2.65rem; } + .compilerCards { grid-template-columns: 1fr; } .boundaryDiagram { padding: 1.1rem; } .hostBox, .languageBox { padding: 1.3rem; } .hostBox strong, .languageBox strong { font-size: 1.8rem; } diff --git a/website/src/pages/index.tsx b/website/src/pages/index.tsx index 2228d85..648586c 100644 --- a/website/src/pages/index.tsx +++ b/website/src/pages/index.tsx @@ -141,13 +141,13 @@ function HomepageHeader() {