Import: share a referenced project's CCU instead of pickling it - #20416
Import: share a referenced project's CCU instead of pickling it#20416auduchinok wants to merge 3 commits into
Conversation
❗ Release notes requiredYou can open this PR in browser to add release notes: open in github.dev
|
befdc7b to
d62a9b1
Compare
|
This is ready. |
d62a9b1 to
cda7c71
Compare
This comment has been minimized.
This comment has been minimized.
T-Gro
left a comment
There was a problem hiding this comment.
🤖 🕵️ AI review — verify independently.
| fslibCcu, | ||
| directoryToResolveRelativePaths, | ||
| isInteractive, | ||
| checkNullness, |
There was a problem hiding this comment.
🤖 🕵️ FS3261 despite --checknulls- after another project warms the same checker: this copy keeps the cached project's checkNullness (both compiler modes).
// First project: --langversion:8.0 --checknulls+
module Warm
let x = 1
// Then, same FSharpChecker:
// --langversion:9.0 --checknulls- --warnaserror+
module Target
let x: string = null
// Warm checker: FS3261. Fresh checker: no diagnostics.| then | ||
| shareable.Add name |> ignore | ||
| match data with | ||
| | :? IImportedProjectCcu when takenFromProject name -> shareableNames.Add name |> ignore |
There was a problem hiding this comment.
🤖 🕵️ Second consumer gets FS0001 (Shared.Carried versus Shared.Carried): cached Middle.dll stays linked to the first consumer's bound copy. Sharing disabled: both pass.
// Dependency.fs: two distinct projects, both named Dependency
module Dependency
type Marker = Marker
// Shared.fs: one project referencing Dependency project 1
module Shared
type Carried = { Value: Dependency.Marker option }
// Middle.fs: compiled DLL referencing Shared.dll and Dependency.dll
module Middle
let carry (x: Shared.Carried) = x
// Check consumer 1, then consumer 2, using the same checker:
// 1 references Shared project + Dependency project 1 + Middle.dll
// 2 references SAME Shared project + Dependency project 2 + SAME Middle.dll
module Consumer
let x: Shared.Carried = Middle.carry { Value = None }| let ctxt = | ||
| mkRemapContextLeavingAssembly g (StackGuard("RemapExprStackGuardDepth")) rescopeAccessTo | ||
|
|
||
| remapTyconToNonLocal ctxt tmenv x |
There was a problem hiding this comment.
🤖 🕵️ FS0001 on a valid consumer extension with shared project references: the exported trait retains the producer's resolution context. Unpickling clears it, letting the consumer supply its own.
// Library.fs, --langversion:preview
module Library
let inline negate (x: ^T) =
(^T : (static member Negate: ^T -> ^T) x)
// Consumer.fs, --langversion:preview; references Library project
module Consumer
type System.Int32 with
static member Negate(x: int) = -x
let value = Library.negate 42
// Sharing on: FS0001, int lacks Negate. Off: no diagnostics.| | true, (already, _) -> already | ||
| | _ -> | ||
| let entry = | ||
| match resolve ccu.AssemblyName with |
There was a problem hiding this comment.
🤖 🕵️ Missing-reference FS0074 disappears: the producer fallback lets the consumer use the public Leaf.Carried type without referencing Leaf. Sharing disabled: FS0074; enabled: no diagnostics.
// Leaf.fs
module Leaf
type Carried = { Value: int }
// Middle.fs, references Leaf project
module Middle
let make () : Leaf.Carried = { Value = 1 }
// Consumer.fs, references Middle project but NOT Leaf
module Consumer
let value = (Middle.make()).Value| let remapping = | ||
| MakeExportRemappingWith ccuRebind viewedCcu generatedCcu.Contents | ||
|
|
||
| let contents = |
There was a problem hiding this comment.
🤖 🕵️ Imported anonymous-record assembly metadata becomes (None, ""): its owner still points to the compiling CCU rather than viewedCcu, for both struct and reference records.
// Library.fs
module Library
let record = {| Value = 1 |} // also: struct {| Value = 1 |}
// Consumer.fs, references Library project
module Consumer
let record = Library.record
// FCS client: apply to Library.record from the referenced assembly
let metadata (value: FSharp.Compiler.Symbols.FSharpMemberOrFunctionOrValue) =
let owner = value.FullType.AnonRecordTypeDetails.Assembly
owner.FileName, owner.QualifiedName
// Sharing on: (None, ""). Off: output path and full assembly identity.| let PruneExportedSignatureInPlace (mspec: ModuleOrNamespace) = | ||
| let rec pruneEntity (entity: Entity) = | ||
| entity.entity_il_repr_cache <- null | ||
| pruneContents entity.ModuleOrNamespaceType |
There was a problem hiding this comment.
🤖 🕵️ MembersFunctionsAndValues returns the default C.M twice: the explicit implementation survives pruning and relinks to it. p_tcaug filters these entries before export.
// Library.fs
module Library
type IFoo =
abstract M: unit -> int
type C() =
abstract M: unit -> int
default _.M() = 2
interface IFoo with
member _.M() = 1
// FCS client: apply to imported Library.C in a referencing project
let members (classType: FSharp.Compiler.Symbols.FSharpEntity) =
classType.MembersFunctionsAndValues
|> Seq.filter (fun m -> m.LogicalName = "M" && not m.IsDispatchSlot)
|> Seq.map (fun m -> m.DisplayName, m.IsExplicitInterfaceImplementation)
|> Seq.toArray
// Sharing on: [|("M", false); ("M", false)|]. Off: [|("M", false)|].09afc51 to
7ee068d
Compare
A project reference pickles its signature once and every consumer unpickles its own copy: five projects referencing one hub hold five copies of its signature TAST. The pickled tree is already in the shape consumers need, so offer it directly through IImportedProjectCcu, and pickle only when something asks. Four things make it usable: - Remapped against a ccu made for this view, not the project's live one, which a consumer would else hold whole. - PruneExportedSignatureInPlace brings it to the shape unpickling produces: no value definitions, no display-only data, no compiled-representation cache. - Every non-local reference is re-pointed at the reading project's ccu of the same name, as unpickling does; a name the reader lacks keeps ours. So a consumer on another framework takes it too. - What each name bound to is recorded, and a second consumer takes that copy only where it resolves all of them alike. Binding runs once the batch is registered and before anything relinks: an assembly unpickled beside it resolves the names it mentions against what is registered, and a delayed CCU is an error there. Depends on sharing imported assemblies (dotnet#20296). Also fixes a check-then-act race in BackgroundCompiler: builders were cached without a second look under the gate, so callers arriving together each built the project. Retained memory, under editor options. Diagnostics identical with the change off and on: Fantomas 8 proj 350.2 -> 275.7 MB -74.5 (-21.3%) ReSharper.FSharp 10 proj 384.9 -> 304.6 MB -80.3 (-20.9%) FsToolkit 8 proj 87.7 -> 70.1 MB -17.6 (-20.1%) Oxpecker 16 proj 131.7 -> 114.2 MB -17.5 (-13.3%) Prime 5 proj 112.2 -> 97.9 MB -14.3 (-12.7%) IcedTasks 7 proj 97.0 -> 92.4 MB -4.6 (-4.7%) FCS solution 14 proj 820.8 -> 818.7 MB -2.1 (-0.2%) consoleapp 1 proj 29.4 -> 29.4 MB 0.0 The FCS solution gains least: nearly every edge in it crosses framework import layers, so each consumer rebuilds a copy of its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
7ee068d to
552a152
Compare
|
🔍 Tooling Safety Check — Affects-Compiler-Output, Affects-Design-Time
|
A project reference pickles its signature once and every consumer unpickles its own copy: five projects
referencing one hub hold five copies of its signature TAST. The pickled tree is already in the shape
consumers need, so offer it directly through
IImportedProjectCcu, and pickle only when something asks.Depends on the shared imported assemblies change (#20296), whose keys let a consumer establish it will hold
the same ccus.
Four things make it usable:
whole.
PruneExportedSignatureInPlacebrings it to the shape unpickling produces: no value definitions, nodisplay-only data, no compiled-representation cache. Rescoping happens in the same traversal that builds
the tree -
ILScopeRef.Localis whatinternalis rooted at, so a consumer would otherwise read thereference's internals as its own.
u_ccurefdoesfor a consumer reading the bytes; a name the reader lacks keeps ours. This is what lets a consumer on
another framework take the contents.
them alike. Consumers that disagree get a copy each, capped at eight.
Binding runs once the batch is registered and before anything relinks: an assembly unpickled beside it
resolves the names it mentions against what is registered, and a delayed CCU is an error there rather than
a wait.
The numbers are smaller due to other FSharpChecker creation flags.
consoleapp has no project references. Not all of this is the handover: switching the handover alone off and
on inside one build moves the compiler's own solution by 2 MB rather than 61, the rest coming from dropping
the language version out of the import reuse key, which lets projects that differ only in it share one
imported graph.
Also fixes a check-then-act race in
BackgroundCompilerthat predates this: builders were cached without asecond look under the gate, so callers arriving together each built the project, and a project built twice
is two of everything it imports. Nothing here survives that.